-- Logs begin at Sun 2022-09-04 00:10:48 UTC, end at Sun 2022-09-04 01:30:06 UTC. -- Sep 04 00:46:53.049329 dv-cinder-master-3-smb systemd[1]: Starting Devstack devstack@keystone.service... Sep 04 00:46:53.056528 dv-cinder-master-3-smb devstack@keystone.service[81618]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 04 00:46:53.060073 dv-cinder-master-3-smb devstack@keystone.service[81618]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Sep 4 00:46:53 2022] *** Sep 04 00:46:53.060073 dv-cinder-master-3-smb devstack@keystone.service[81618]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 04 00:46:53.060073 dv-cinder-master-3-smb devstack@keystone.service[81618]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 04 00:46:53.060073 dv-cinder-master-3-smb devstack@keystone.service[81618]: nodename: dv-cinder-master-3-smb Sep 04 00:46:53.060073 dv-cinder-master-3-smb devstack@keystone.service[81618]: machine: x86_64 Sep 04 00:46:53.060073 dv-cinder-master-3-smb devstack@keystone.service[81618]: clock source: unix Sep 04 00:46:53.060073 dv-cinder-master-3-smb devstack@keystone.service[81618]: pcre jit disabled Sep 04 00:46:53.060073 dv-cinder-master-3-smb devstack@keystone.service[81618]: detected number of CPU cores: 4 Sep 04 00:46:53.060073 dv-cinder-master-3-smb devstack@keystone.service[81618]: current working directory: / Sep 04 00:46:53.060073 dv-cinder-master-3-smb devstack@keystone.service[81618]: detected binary path: /usr/bin/uwsgi-core Sep 04 00:46:53.060073 dv-cinder-master-3-smb devstack@keystone.service[81618]: your processes number limit is 47680 Sep 04 00:46:53.060073 dv-cinder-master-3-smb devstack@keystone.service[81618]: your memory page size is 4096 bytes Sep 04 00:46:53.060073 dv-cinder-master-3-smb devstack@keystone.service[81618]: detected max file descriptor number: 2048 Sep 04 00:46:53.060073 dv-cinder-master-3-smb devstack@keystone.service[81618]: lock engine: pthread robust mutexes Sep 04 00:46:53.060073 dv-cinder-master-3-smb devstack@keystone.service[81618]: thunder lock: enabled Sep 04 00:46:53.060073 dv-cinder-master-3-smb devstack@keystone.service[81618]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 04 00:46:53.060073 dv-cinder-master-3-smb devstack@keystone.service[81618]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 04 00:46:53.098549 dv-cinder-master-3-smb devstack@keystone.service[81618]: Python main interpreter initialized at 0x55c2cd1db9c0 Sep 04 00:46:53.098549 dv-cinder-master-3-smb devstack@keystone.service[81618]: python threads support enabled Sep 04 00:46:53.098549 dv-cinder-master-3-smb devstack@keystone.service[81618]: your server socket listen backlog is limited to 100 connections Sep 04 00:46:53.098549 dv-cinder-master-3-smb devstack@keystone.service[81618]: your mercy for graceful operations on workers is 90 seconds Sep 04 00:46:53.098871 dv-cinder-master-3-smb devstack@keystone.service[81618]: mapped 403077 bytes (393 KB) for 2 cores Sep 04 00:46:53.098967 dv-cinder-master-3-smb devstack@keystone.service[81618]: *** Operational MODE: preforking *** Sep 04 00:46:53.099240 dv-cinder-master-3-smb devstack@keystone.service[81618]: *** uWSGI is running in multiple interpreter mode *** Sep 04 00:46:53.099240 dv-cinder-master-3-smb devstack@keystone.service[81618]: spawned uWSGI master process (pid: 81618) Sep 04 00:46:53.099448 dv-cinder-master-3-smb systemd[1]: Started Devstack devstack@keystone.service. Sep 04 00:46:53.100261 dv-cinder-master-3-smb devstack@keystone.service[81618]: spawned uWSGI worker 1 (pid: 81629, cores: 1) Sep 04 00:46:53.103481 dv-cinder-master-3-smb devstack@keystone.service[81618]: spawned uWSGI worker 2 (pid: 81630, cores: 1) Sep 04 00:46:53.103481 dv-cinder-master-3-smb devstack@keystone.service[81618]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 04 00:46:57.449552 dv-cinder-master-3-smb devstack@keystone.service[81630]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 04 00:46:57.449552 dv-cinder-master-3-smb devstack@keystone.service[81630]: from cryptography.utils import int_from_bytes Sep 04 00:46:57.450336 dv-cinder-master-3-smb devstack@keystone.service[81630]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 04 00:46:57.450336 dv-cinder-master-3-smb devstack@keystone.service[81630]: from cryptography.utils import int_from_bytes Sep 04 00:46:57.451100 dv-cinder-master-3-smb devstack@keystone.service[81629]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 04 00:46:57.451100 dv-cinder-master-3-smb devstack@keystone.service[81629]: from cryptography.utils import int_from_bytes Sep 04 00:46:57.451538 dv-cinder-master-3-smb devstack@keystone.service[81629]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 04 00:46:57.451538 dv-cinder-master-3-smb devstack@keystone.service[81629]: from cryptography.utils import int_from_bytes Sep 04 00:46:57.507604 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 04 00:46:57.507818 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 04 00:46:57.507951 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 04 00:46:57.508120 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] command line args: [] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 04 00:46:57.508232 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 04 00:46:57.508232 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] command line args: [] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 04 00:46:57.508488 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 04 00:46:57.508488 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ================================================================================ {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 04 00:46:57.508734 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 04 00:46:57.508734 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ================================================================================ {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 04 00:46:57.508948 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] admin_token = **** {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.509067 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] admin_token = **** {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.509191 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] config_dir = [] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.509191 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.509427 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] config_dir = [] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.509544 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] config_source = [] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.509666 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.510041 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] debug = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.510206 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] config_source = [] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.510370 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.511809 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] debug = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.511809 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.512108 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.512250 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.512375 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.512539 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.512670 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.512848 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.512965 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.513147 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.513267 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] list_limit = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.513448 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] list_limit = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.513554 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] log_config_append = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.513685 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] log_config_append = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.513814 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.513931 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.514142 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] log_dir = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.514320 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] log_dir = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.514444 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] log_file = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.514625 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] log_file = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.514731 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.514902 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.515023 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.515195 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.515308 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.515481 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.515589 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.515733 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.515851 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.516024 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.516152 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.516281 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.516396 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.516597 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.516707 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.516869 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.516988 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.517148 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.517148 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.517372 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.517372 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.517597 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.517711 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.517824 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.517966 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.518087 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.518194 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.518314 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.518430 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.518628 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.518765 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.518882 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.518991 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] publish_errors = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.519117 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] publish_errors = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.519223 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.519341 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.519456 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.519571 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.519745 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.519861 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.519983 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.520175 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.520291 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.520423 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.520592 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] standard_threads = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.520699 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] standard_threads = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.520818 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.520924 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.521044 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.521164 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.521279 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.521530 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.521793 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] use_journal = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.521901 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] use_journal = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.521901 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] use_json = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.522134 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] use_json = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.522252 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] use_stderr = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.522842 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] use_stderr = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.523197 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] use_syslog = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.523492 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] use_syslog = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.523582 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.523732 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:46:57.523732 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.523873 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.523945 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.524023 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.524173 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.524255 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.524328 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.524398 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.524537 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.524616 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.524696 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.524775 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.524847 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.524918 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.524994 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.525065 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.525143 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.525216 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.525286 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.525363 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.525441 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.525512 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.525607 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.525725 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.525804 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.525883 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.525965 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.526041 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.526121 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.526194 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.526264 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.526335 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.526411 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.526483 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.526561 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.526639 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.526710 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.526785 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.526857 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.526926 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.527004 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.527088 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.527174 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.527246 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.527316 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.527393 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.527476 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.527548 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.527618 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.527688 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.527757 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.527833 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.528041 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.528124 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.528202 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.528275 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.528351 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.528429 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.530010 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.530122 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.530220 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.530306 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.530380 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.530458 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.530537 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.530609 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.530679 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.530749 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.530825 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.530907 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.530980 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.531050 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.531126 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.531203 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.531280 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.531359 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.531432 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.531508 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.531585 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.531662 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.531734 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.531811 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.531890 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.531962 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.532031 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.532101 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.532204 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.532289 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.532362 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.532502 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.532609 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.532708 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.532815 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.532930 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.533030 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.533127 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.533231 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.533339 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.533438 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.533544 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.533637 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.533712 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.533840 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.533958 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.534034 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.534150 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.534234 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.534309 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.534433 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.534511 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.534582 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.534698 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.534789 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.534865 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.534982 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.535057 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.535165 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.535244 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.535324 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.535398 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.535515 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.535590 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.535706 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.535782 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.535866 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.535940 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.536065 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.536150 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.536224 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.536340 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.536415 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.536564 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.536648 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.536721 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.536841 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.536916 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.537032 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.537120 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.537196 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.537312 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.537387 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.537458 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.537574 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.537658 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.537738 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.537811 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.538039 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.538125 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.538315 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.539140 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.539231 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.539363 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.539450 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.539524 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.539604 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.539678 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.539749 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.539820 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.539891 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.539962 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.540090 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.540179 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.540310 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.540388 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.540477 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.540551 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.540623 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.540706 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.540787 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.540861 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.540861 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.540990 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.541060 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.541137 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.541217 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.541296 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.541368 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.541439 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.541510 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.541580 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.541659 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.541732 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.541803 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.541874 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.541945 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.542016 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.542098 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.542171 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.542242 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.542313 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.542384 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.542455 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.542534 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.542613 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.542685 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.542757 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.542827 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.542898 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.542977 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.543050 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.543121 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.543200 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.543274 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.543344 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.543423 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.543501 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.543574 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.543645 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.543791 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.543874 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.544310 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.544405 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.544627 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.544743 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.544876 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] auth.external = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.544956 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] auth.external = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.545059 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.545142 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.545353 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.545436 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.545583 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.545663 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.545816 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] auth.password = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.545910 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] auth.password = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.546049 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] auth.token = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.546130 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] auth.token = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.546292 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.546372 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.546532 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.546621 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.546781 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.546861 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.547017 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.547097 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.547265 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.547355 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.547512 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.547592 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.547750 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.547828 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.547985 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] credential.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.548072 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] credential.caching = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.548231 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.548316 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.548494 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.548576 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.548749 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.548838 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.548992 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.549073 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.549237 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.549318 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.549474 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.549562 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.549716 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.549797 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.549970 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.550049 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.550205 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.550292 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.550516 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.550604 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.550737 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.550817 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.550983 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.551154 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.551290 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.551369 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.551480 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.551562 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.551738 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.551828 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.551977 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] federation.caching = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.552065 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] federation.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.552211 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.552291 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.552482 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.552579 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.552723 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.552802 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.552953 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.553032 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.553198 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.553286 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.553434 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.553515 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.553735 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.553845 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.553968 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.554062 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.554230 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.554409 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.554484 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.554556 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.554736 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.554828 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.554977 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.555058 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] identity.caching = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.555225 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.555320 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.555473 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.555562 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.555715 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.555796 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.555955 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.556035 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.556205 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.556286 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.556436 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.556539 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.556696 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.556779 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.556946 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.557027 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.557194 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.557273 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.557425 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.557505 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.557656 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.557738 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.557886 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.557973 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.558128 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.558208 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.558373 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.558454 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.558616 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.558697 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.558873 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.558955 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.559117 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.559198 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.559388 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.559467 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.559624 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.559702 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.559865 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.559946 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.560095 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.560174 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.560334 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.560413 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.560585 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.560666 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.560822 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.560902 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.561063 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.561144 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.561302 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.561386 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.561547 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.561627 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.561778 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.561857 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.562023 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.562104 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.562269 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.562349 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.562504 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.562583 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.562748 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.562835 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.562995 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.563078 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.563231 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.563311 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.563474 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.563556 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.563703 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.563781 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.563940 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.564020 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.564178 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.564260 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.564419 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.564519 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.564682 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.564762 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.564920 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.565001 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.565167 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.565247 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.565527 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.565608 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.565761 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.565842 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.565995 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.566079 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.566245 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.566326 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.566495 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.566575 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.566731 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.566811 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.566970 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.567049 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.567207 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.567288 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.567440 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.567519 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.567679 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.567758 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.567939 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.568026 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.568170 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.568249 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.568417 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.568594 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.568672 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.568743 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.568924 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.569007 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.569164 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.569298 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.569474 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.569636 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.569711 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.569784 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.569897 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.569986 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.570133 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.570214 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.570363 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.570444 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.570608 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.570697 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.570853 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.570933 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.571097 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.571177 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.571341 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.571431 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.571584 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.571665 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.571814 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.571894 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.572058 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.572147 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.572297 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.572384 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.572553 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.572635 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.572791 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.575992 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.575992 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.576192 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.576192 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.576300 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.576300 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.576439 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.576535 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.576610 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.576610 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.576734 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.576805 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.576947 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.577024 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.577096 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.577168 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.577239 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.577310 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.577390 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.577464 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.577535 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.577606 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.577678 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.577750 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.577829 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.577903 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.577974 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.577974 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.578097 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.578168 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] resource.caching = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.578247 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] resource.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.578321 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.578393 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.578482 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.578563 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.578637 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.578706 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.578768 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.578826 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.578897 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.578963 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.579033 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.579111 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.579185 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.579257 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.579328 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.579399 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.579471 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.579550 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.579623 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] role.caching = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.579694 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] role.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.579765 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] role.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.579835 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] role.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.579901 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.579979 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.580052 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.580123 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.580200 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.580273 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.580345 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.580424 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.580515 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.580577 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.580641 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.580712 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.580800 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.580876 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.580942 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.581013 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.581085 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.581156 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.581235 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.581308 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.581379 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.581488 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.581569 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.581772 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.581862 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.582023 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.582102 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.582266 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.582430 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.582663 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.582752 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.582899 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.582980 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.583147 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.583225 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.583396 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.583484 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.583639 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.583718 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.583873 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.583952 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.584106 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.584193 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.584358 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.584445 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.584623 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.584703 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.585383 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.585487 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.585619 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.585688 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.585856 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.585967 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.586079 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.586167 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.586321 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.586404 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.586574 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.586654 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.586809 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.586890 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.587041 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.587127 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.587271 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.587350 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.587496 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.587576 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] token.caching = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.587731 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] token.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.587812 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.587968 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.588048 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.588212 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.588301 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.588491 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.588600 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.588752 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.588904 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.589004 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.589081 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.589223 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.589305 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.589470 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.589550 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.589719 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.589801 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.589958 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.590039 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.590191 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.590269 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.590426 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.590507 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.590659 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.590739 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.590905 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.590992 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.591157 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.591246 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.591391 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.591481 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.591633 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.591715 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.591896 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.591976 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] database.connection = **** {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.592139 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.592227 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] database.connection = **** {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.592382 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.592481 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.592653 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.592734 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.592888 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.592981 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.593124 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.593204 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.593354 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.593433 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.593590 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.593749 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.593827 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.593901 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.594062 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.594143 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.594300 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.594390 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.594532 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.594612 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.594773 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.594854 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.595006 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.595095 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.595243 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.595323 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.595474 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.595552 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.595711 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.595799 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.595936 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.596016 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.596175 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.596256 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.596328 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 04 00:46:57.596408 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:46:57.596586 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 04 00:46:57.626122 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.626357 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.626482 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.626592 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.626715 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.626938 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.635207 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.635508 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.665841 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.666069 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.666158 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.666352 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.722035 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.722320 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.779026 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.779202 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.780213 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.780448 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.950475 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.950714 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.950869 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.950961 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.951102 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.951189 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.951334 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.951474 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.951562 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.951642 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.951759 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.951979 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.979190 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.979340 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.979429 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.979575 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.979674 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.979800 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.979885 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.980022 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.980155 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.980233 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.980317 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.980433 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.980540 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.980697 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.992314 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:57.992596 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:58.008301 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:58.008500 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:58.008606 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:58.008743 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:58.008827 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:58.008969 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:58.009054 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:58.009184 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:58.009266 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:58.009400 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:58.009492 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:58.009612 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:58.009695 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:58.009826 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 04 00:46:58.026189 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.026398 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 04 00:46:58.026483 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.026610 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 04 00:46:58.026743 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.026945 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.027029 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 04 00:46:58.027144 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 04 00:46:58.027230 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.027411 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.027494 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.027669 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.027760 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.027927 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.028009 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.028183 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.037632 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.037867 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.038488 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.038654 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.038739 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.038922 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.039004 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.039205 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.046212 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.046554 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.049151 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.049441 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.049541 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.049728 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.049813 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.050016 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.050100 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.050294 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.050391 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.050558 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.050642 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.050830 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.050912 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.051107 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.051190 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.051380 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.051537 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.051659 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.051751 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.051931 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.052014 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.052206 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.075349 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.075929 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.076855 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.077147 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.077415 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.077699 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.081815 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.082102 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.082388 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.082470 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.082743 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.083022 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.089307 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.089603 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.090556 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.090843 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.095272 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.095569 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.095862 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.096153 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.096442 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.096605 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.096737 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.101776 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.101912 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.101992 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.102055 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.102128 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.102128 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.102128 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.102128 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.102128 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.112789 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.113080 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.114466 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.114747 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.117064 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.118698 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.120290 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.120614 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.122168 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.122467 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.124830 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.126774 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.127941 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.129969 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.131222 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.131526 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.132336 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.132667 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.132954 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.133255 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.134045 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.134355 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.134642 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.134925 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.135201 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.135485 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.150175 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.150468 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.150738 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.151005 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.152128 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.152420 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.152721 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.152983 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.157873 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.159427 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.159956 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.160975 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.161347 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.161619 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.163164 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.163439 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.164044 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.164343 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.164664 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.164955 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.165347 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.166165 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.170618 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.170618 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.170618 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.170618 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.176761 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.177053 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.177356 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.177629 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.178017 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.178305 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.178614 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.179038 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.179321 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.179602 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.179879 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.180256 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.180559 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.180838 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.193992 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.196801 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.197648 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.199794 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.200553 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.202013 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.202759 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.203520 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.203816 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.204223 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.204306 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.205733 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.206077 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.206474 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.211343 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.212842 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.213723 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.215182 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.215763 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.216166 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.216477 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.216761 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.218121 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.218516 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.218805 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.219090 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.224638 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.224948 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.225248 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.227978 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.227978 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.227978 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.232281 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.232594 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.232883 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.233163 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.233440 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.233727 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.234006 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.234287 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.234563 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.234716 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 04 00:46:58.234847 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.235003 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.235142 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.235283 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.235415 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.235624 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.235718 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.235902 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.236035 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.236187 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.236481 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.236762 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.237029 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.237315 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.237596 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.237859 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.238152 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.238415 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 04 00:46:58.272057 dv-cinder-master-3-smb devstack@keystone.service[81630]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55c2cd1db9c0 pid: 81630 (default app) Sep 04 00:46:58.273807 dv-cinder-master-3-smb devstack@keystone.service[81629]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55c2cd1db9c0 pid: 81629 (default app) Sep 04 00:46:58.275908 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda466a9-4416-48dd-9664-cee42ffbb7ce None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:46:58.276200 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda466a9-4416-48dd-9664-cee42ffbb7ce None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:46:58.276495 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda466a9-4416-48dd-9664-cee42ffbb7ce None None] PATH_INFO: `/v3` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:46:58.280688 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1/1] 10.222.0.27 () {50 vars in 791 bytes} [Sun Sep 4 00:46:58 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:05.619465 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f01ab-2cfb-4fc6-bdc8-2bf373e651af None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:05.619981 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f01ab-2cfb-4fc6-bdc8-2bf373e651af None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:05.619981 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f01ab-2cfb-4fc6-bdc8-2bf373e651af None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:05.624269 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1/2] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:05 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 00:47:05.629496 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d268081-6040-402c-bd72-1e37cf300f65 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:05.629960 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d268081-6040-402c-bd72-1e37cf300f65 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:05.630056 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d268081-6040-402c-bd72-1e37cf300f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:05.701078 dv-cinder-master-3-smb devstack@keystone.service[81630]: INFO dbcounter [None req-1d268081-6040-402c-bd72-1e37cf300f65 None None] Registered counter for database keystone Sep 04 00:47:05.719862 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG oslo_db.sqlalchemy.engines [None req-1d268081-6040-402c-bd72-1e37cf300f65 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81630) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 04 00:47:05.726590 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writer thread running {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 04 00:47:05.908379 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG passlib.handlers.bcrypt [None req-1d268081-6040-402c-bd72-1e37cf300f65 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81630) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 04 00:47:05.908730 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG passlib.handlers.bcrypt [None req-1d268081-6040-402c-bd72-1e37cf300f65 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81630) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 04 00:47:05.966857 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-1d268081-6040-402c-bd72-1e37cf300f65 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:05.968494 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-1d268081-6040-402c-bd72-1e37cf300f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:06.096062 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 2/3] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:05 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 04 00:47:06.561195 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f01ab-2cfb-4fc6-bdc8-2bf373e651af None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:06.629956 dv-cinder-master-3-smb devstack@keystone.service[81629]: INFO dbcounter [None req-a51f01ab-2cfb-4fc6-bdc8-2bf373e651af None None] Registered counter for database keystone Sep 04 00:47:06.651865 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG oslo_db.sqlalchemy.engines [None req-a51f01ab-2cfb-4fc6-bdc8-2bf373e651af None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81629) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 04 00:47:06.658095 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writer thread running {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 04 00:47:06.776187 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f01ab-2cfb-4fc6-bdc8-2bf373e651af None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:06.780404 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b1be5-81e5-4c87-b656-8939a5c5ecc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:06.781074 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b1be5-81e5-4c87-b656-8939a5c5ecc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:06.781352 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b1be5-81e5-4c87-b656-8939a5c5ecc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:06.781621 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b1be5-81e5-4c87-b656-8939a5c5ecc6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:06.790242 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b1be5-81e5-4c87-b656-8939a5c5ecc6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:06.841969 dv-cinder-master-3-smb devstack@keystone.service[81629]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d48b1be5-81e5-4c87-b656-8939a5c5ecc6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 04 00:47:06.842258 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b1be5-81e5-4c87-b656-8939a5c5ecc6 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:06.852016 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 2/4] 10.222.0.27 () {58 vars in 1324 bytes} [Sun Sep 4 00:47:06 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 04 00:47:06.857999 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d268081-6040-402c-bd72-1e37cf300f65 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:06.872768 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d268081-6040-402c-bd72-1e37cf300f65 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:06.877126 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57a2f1e-9d1a-4e8e-ae44-613ff74be76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:06.878214 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a2f1e-9d1a-4e8e-ae44-613ff74be76f admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:06.878494 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a2f1e-9d1a-4e8e-ae44-613ff74be76f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:06.878766 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a2f1e-9d1a-4e8e-ae44-613ff74be76f admin admin] PATH_INFO: `/v3/projects/a1d8c40dd6c04e8e8fe6b0c2b34723cd` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:06.880328 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57a2f1e-9d1a-4e8e-ae44-613ff74be76f admin admin] RBAC: Authorizing `identity:get_project(project_id=a1d8c40dd6c04e8e8fe6b0c2b34723cd)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:06.933256 dv-cinder-master-3-smb devstack@keystone.service[81630]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d57a2f1e-9d1a-4e8e-ae44-613ff74be76f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 04 00:47:06.933604 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57a2f1e-9d1a-4e8e-ae44-613ff74be76f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:06.938270 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 3/5] 10.222.0.27 () {56 vars in 1207 bytes} [Sun Sep 4 00:47:06 2022] GET /identity/v3/projects/a1d8c40dd6c04e8e8fe6b0c2b34723cd => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:08.814018 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87540ee6-1c03-4c0f-a4e2-3e65beeb264c None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:08.814531 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87540ee6-1c03-4c0f-a4e2-3e65beeb264c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:08.814531 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87540ee6-1c03-4c0f-a4e2-3e65beeb264c None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:08.818651 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 3/6] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 00:47:08.824126 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b5a9f-92fd-46ce-83ca-6e6fff2ddfbc None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:08.824531 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b5a9f-92fd-46ce-83ca-6e6fff2ddfbc None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:08.824712 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b5a9f-92fd-46ce-83ca-6e6fff2ddfbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:08.845193 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-2f4b5a9f-92fd-46ce-83ca-6e6fff2ddfbc None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:08.846719 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2f4b5a9f-92fd-46ce-83ca-6e6fff2ddfbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:08.855456 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 4/7] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:08 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 04 00:47:09.130016 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87540ee6-1c03-4c0f-a4e2-3e65beeb264c None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:09.142137 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87540ee6-1c03-4c0f-a4e2-3e65beeb264c None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:09.146320 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281509e-528b-4b8e-b473-29c41e651d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:09.146932 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281509e-528b-4b8e-b473-29c41e651d8e admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:09.147220 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281509e-528b-4b8e-b473-29c41e651d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:09.147486 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281509e-528b-4b8e-b473-29c41e651d8e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:09.155094 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3281509e-528b-4b8e-b473-29c41e651d8e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:09.156902 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3281509e-528b-4b8e-b473-29c41e651d8e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:09.166158 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 4/8] 10.222.0.27 () {58 vars in 1324 bytes} [Sun Sep 4 00:47:09 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 04 00:47:09.171640 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b5a9f-92fd-46ce-83ca-6e6fff2ddfbc None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:09.181559 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b5a9f-92fd-46ce-83ca-6e6fff2ddfbc None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:09.185880 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07db23-fdd7-48ca-bf27-9fbf8e71d265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:09.186504 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07db23-fdd7-48ca-bf27-9fbf8e71d265 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:09.186783 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07db23-fdd7-48ca-bf27-9fbf8e71d265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:09.187057 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07db23-fdd7-48ca-bf27-9fbf8e71d265 admin admin] PATH_INFO: `/v3/users/644f0b1b120742d2aca15aeeaa76abcc` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:09.188344 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b07db23-fdd7-48ca-bf27-9fbf8e71d265 admin admin] RBAC: Authorizing `identity:get_user(user_id=644f0b1b120742d2aca15aeeaa76abcc)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:09.190194 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b07db23-fdd7-48ca-bf27-9fbf8e71d265 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:09.191694 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 5/9] 10.222.0.27 () {56 vars in 1198 bytes} [Sun Sep 4 00:47:09 2022] GET /identity/v3/users/644f0b1b120742d2aca15aeeaa76abcc => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:11.153091 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d6230-bfca-4763-baf9-2d615141f480 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:11.153723 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d6230-bfca-4763-baf9-2d615141f480 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:11.153723 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d6230-bfca-4763-baf9-2d615141f480 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:11.154961 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 5/10] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:11.160056 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5bc3cd-a40f-4c7b-8d0a-3be87ea8a638 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:11.166271 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5bc3cd-a40f-4c7b-8d0a-3be87ea8a638 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:11.166271 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5bc3cd-a40f-4c7b-8d0a-3be87ea8a638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:11.179509 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-ce5bc3cd-a40f-4c7b-8d0a-3be87ea8a638 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:11.180736 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ce5bc3cd-a40f-4c7b-8d0a-3be87ea8a638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:11.206338 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628af030-6e40-49b1-a003-24c0b09da170 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:11.206621 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628af030-6e40-49b1-a003-24c0b09da170 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:11.206894 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628af030-6e40-49b1-a003-24c0b09da170 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:11.208205 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 6/11] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:11.213144 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e6919c-bb3e-4d71-8adb-d24efac3ab95 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:11.213433 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e6919c-bb3e-4d71-8adb-d24efac3ab95 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:11.213713 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e6919c-bb3e-4d71-8adb-d24efac3ab95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:11.255545 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG passlib.handlers.bcrypt [None req-c7e6919c-bb3e-4d71-8adb-d24efac3ab95 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81629) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 04 00:47:11.256511 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG passlib.handlers.bcrypt [None req-c7e6919c-bb3e-4d71-8adb-d24efac3ab95 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81629) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 04 00:47:11.264716 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 6/12] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:11.306348 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c7e6919c-bb3e-4d71-8adb-d24efac3ab95 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:11.307908 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c7e6919c-bb3e-4d71-8adb-d24efac3ab95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:11.404872 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 7/13] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 00:47:11.539620 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5bc3cd-a40f-4c7b-8d0a-3be87ea8a638 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:11.566237 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:11.643804 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5bc3cd-a40f-4c7b-8d0a-3be87ea8a638 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:11.671880 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e6919c-bb3e-4d71-8adb-d24efac3ab95 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:11.680784 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52dd5ce-a8ec-48f6-99bd-1ec58fd99404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:11.681641 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52dd5ce-a8ec-48f6-99bd-1ec58fd99404 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:11.681924 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52dd5ce-a8ec-48f6-99bd-1ec58fd99404 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:11.682204 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52dd5ce-a8ec-48f6-99bd-1ec58fd99404 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:11.694285 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52dd5ce-a8ec-48f6-99bd-1ec58fd99404 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:47:11.703199 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-b52dd5ce-a8ec-48f6-99bd-1ec58fd99404 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:47:11.703199 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:11.703199 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:11.703199 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:11.703199 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:11.703199 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:11.703199 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:11.703199 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:11.703199 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:11.703199 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:11.703199 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:11.703199 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:11.703199 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:47:11.703199 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:47:11.703199 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:47:11.703199 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:47:11.704216 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:47:11.704216 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:47:11.704216 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:47:11.704216 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:47:11.704216 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:11.704216 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:11.704216 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:11.704216 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:11.704216 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:11.704216 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:11.704216 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:11.704216 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:11.704216 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:11.704216 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:11.704216 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:11.704216 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:11.705089 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:11.705089 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:11.705089 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:11.705089 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:11.705089 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:47:11.705089 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:47:11.705089 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:47:11.705089 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:47:11.705089 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:47:11.705089 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:47:11.705089 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:47:11.705089 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:47:11.705089 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 7/14] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:47:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:47:11.710762 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52dd5ce-a8ec-48f6-99bd-1ec58fd99404 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:11.776795 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e6919c-bb3e-4d71-8adb-d24efac3ab95 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:11.779064 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:11.794679 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:11.813034 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52dd5ce-a8ec-48f6-99bd-1ec58fd99404 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:11.813942 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6c949-c21a-47d5-a257-512eab1ae033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:11.814870 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6c949-c21a-47d5-a257-512eab1ae033 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:11.815153 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6c949-c21a-47d5-a257-512eab1ae033 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:11.815431 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6c949-c21a-47d5-a257-512eab1ae033 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:11.848767 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d6c52-a802-4cd6-baf2-80c223cb39b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:11.849365 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d6c52-a802-4cd6-baf2-80c223cb39b7 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:11.850274 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d6c52-a802-4cd6-baf2-80c223cb39b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:11.850968 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d6c52-a802-4cd6-baf2-80c223cb39b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:11.851643 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16d6c52-a802-4cd6-baf2-80c223cb39b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:11.851949 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16d6c52-a802-4cd6-baf2-80c223cb39b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:11.853370 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16d6c52-a802-4cd6-baf2-80c223cb39b7 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:11.864806 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 8/15] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:47:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:47:11.870929 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d6c52-a802-4cd6-baf2-80c223cb39b7 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:11.885955 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e6c949-c21a-47d5-a257-512eab1ae033 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:11.887608 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e6c949-c21a-47d5-a257-512eab1ae033 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:11.969876 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d6c52-a802-4cd6-baf2-80c223cb39b7 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:11.985666 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 8/16] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:47:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 04 00:47:11.991559 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6c949-c21a-47d5-a257-512eab1ae033 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:11.995132 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:12.005494 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb9136-3487-4d73-bb45-c42e8f6f317a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:12.006131 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb9136-3487-4d73-bb45-c42e8f6f317a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:12.006411 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb9136-3487-4d73-bb45-c42e8f6f317a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:12.006678 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb9136-3487-4d73-bb45-c42e8f6f317a None admin] PATH_INFO: `/v3/users/644f0b1b120742d2aca15aeeaa76abcc` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:12.007807 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb9136-3487-4d73-bb45-c42e8f6f317a None admin] RBAC: Authorizing `identity:get_user(user_id=644f0b1b120742d2aca15aeeaa76abcc)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:12.009315 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb9136-3487-4d73-bb45-c42e8f6f317a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:12.010869 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 9/17] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:47:11 2022] GET /identity/v3/users/644f0b1b120742d2aca15aeeaa76abcc => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:47:12.017011 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb9136-3487-4d73-bb45-c42e8f6f317a None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:12.033489 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:12.093733 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6c949-c21a-47d5-a257-512eab1ae033 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:12.119169 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb9136-3487-4d73-bb45-c42e8f6f317a None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:12.127456 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f45c2-c180-484d-bcda-706590601306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:12.128707 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f45c2-c180-484d-bcda-706590601306 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:12.128989 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f45c2-c180-484d-bcda-706590601306 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:12.129268 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f45c2-c180-484d-bcda-706590601306 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:12.148674 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8f45c2-c180-484d-bcda-706590601306 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:12.150143 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8f45c2-c180-484d-bcda-706590601306 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:12.158018 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de1c6f-09e1-44e7-bc49-46264c4c2324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:12.158651 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de1c6f-09e1-44e7-bc49-46264c4c2324 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:12.158935 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de1c6f-09e1-44e7-bc49-46264c4c2324 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:12.159217 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de1c6f-09e1-44e7-bc49-46264c4c2324 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:12.160338 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de1c6f-09e1-44e7-bc49-46264c4c2324 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:12.161828 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de1c6f-09e1-44e7-bc49-46264c4c2324 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:12.163614 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 10/18] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:47:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:47:12.169170 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-4c8f45c2-c180-484d-bcda-706590601306 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:47:12.169170 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:12.169170 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:12.169170 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:12.169170 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:12.169170 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:12.169170 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:12.169170 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:12.169170 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:12.169170 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:12.169170 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:12.169170 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:12.169170 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:47:12.169170 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:47:12.169170 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:47:12.169170 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:47:12.170356 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:12.170356 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:12.170356 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:12.170356 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:12.170356 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:12.170356 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:12.170356 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:12.170356 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:12.170356 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:12.170356 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:12.170356 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:12.170356 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:12.170356 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:12.170356 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:12.170356 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:12.170356 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:12.170356 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:47:12.171139 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:47:12.171139 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:47:12.171139 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:47:12.171301 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de1c6f-09e1-44e7-bc49-46264c4c2324 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:12.171371 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 9/19] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:47:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:47:12.177188 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f45c2-c180-484d-bcda-706590601306 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:12.234298 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:12.252542 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:12.272817 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de1c6f-09e1-44e7-bc49-46264c4c2324 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:12.278565 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f45c2-c180-484d-bcda-706590601306 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:12.307283 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7d215-4831-4742-b784-84415aeacbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:12.308112 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7d215-4831-4742-b784-84415aeacbe1 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:12.308394 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7d215-4831-4742-b784-84415aeacbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:12.308697 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7d215-4831-4742-b784-84415aeacbe1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:12.309422 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce7d215-4831-4742-b784-84415aeacbe1 None admin] RBAC: Adding query filter params (role.id=c8a59d34ee2f4cefa6940cac778260c7, scope.domain.id=default, user.id=644f0b1b120742d2aca15aeeaa76abcc) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:12.309730 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce7d215-4831-4742-b784-84415aeacbe1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:12.311175 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce7d215-4831-4742-b784-84415aeacbe1 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:12.311677 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d341184-54b7-4da7-9ec4-71bfccce89b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:12.312277 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d341184-54b7-4da7-9ec4-71bfccce89b5 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:12.312589 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d341184-54b7-4da7-9ec4-71bfccce89b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:12.312865 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d341184-54b7-4da7-9ec4-71bfccce89b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:12.313494 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d341184-54b7-4da7-9ec4-71bfccce89b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:12.313790 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d341184-54b7-4da7-9ec4-71bfccce89b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:12.315145 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d341184-54b7-4da7-9ec4-71bfccce89b5 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:12.317664 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 11/20] 10.222.0.27 () {56 vars in 1322 bytes} [Sun Sep 4 00:47:12 2022] GET /identity/v3/role_assignments?user.id=644f0b1b120742d2aca15aeeaa76abcc&scope.domain.id=default&role.id=c8a59d34ee2f4cefa6940cac778260c7 => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:47:12.334917 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 10/21] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:14.375619 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d30a2-e315-4a1f-8dfd-1826daa63c65 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:14.376481 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d30a2-e315-4a1f-8dfd-1826daa63c65 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:14.376926 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d30a2-e315-4a1f-8dfd-1826daa63c65 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:14.378452 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 12/22] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:14.384699 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3a9b4-d66b-4cbb-8f40-45b46def9575 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:14.385122 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3a9b4-d66b-4cbb-8f40-45b46def9575 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:14.385311 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3a9b4-d66b-4cbb-8f40-45b46def9575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:14.409472 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-08d3a9b4-d66b-4cbb-8f40-45b46def9575 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:14.410910 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-08d3a9b4-d66b-4cbb-8f40-45b46def9575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:14.510121 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 11/23] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:14.535643 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d47fa-264d-4fe4-9296-d54d516d8a03 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:14.536260 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d47fa-264d-4fe4-9296-d54d516d8a03 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:14.536832 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d47fa-264d-4fe4-9296-d54d516d8a03 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:14.539268 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 13/24] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:14.545053 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a56e7-0049-4cbf-97c1-0021fd748e2b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:14.545340 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a56e7-0049-4cbf-97c1-0021fd748e2b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:14.545612 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a56e7-0049-4cbf-97c1-0021fd748e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:14.573843 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-fc2a56e7-0049-4cbf-97c1-0021fd748e2b None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:14.575048 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-fc2a56e7-0049-4cbf-97c1-0021fd748e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:14.638416 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:14.665845 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 12/25] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:14.898914 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d47fa-264d-4fe4-9296-d54d516d8a03 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:15.021040 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d47fa-264d-4fe4-9296-d54d516d8a03 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:15.068808 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b61dd3-82ba-48d1-bc47-bf04e645b50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:15.070101 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b61dd3-82ba-48d1-bc47-bf04e645b50e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:15.070586 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b61dd3-82ba-48d1-bc47-bf04e645b50e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:15.071053 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b61dd3-82ba-48d1-bc47-bf04e645b50e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:15.089527 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a11c0-59c8-4a4c-bf35-93573f28dba8 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:15.090147 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a11c0-59c8-4a4c-bf35-93573f28dba8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:15.090609 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a11c0-59c8-4a4c-bf35-93573f28dba8 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:15.092172 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 13/26] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 00:47:15.096858 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b61dd3-82ba-48d1-bc47-bf04e645b50e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:47:15.097674 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa89a80-16a9-49f7-bda3-ab3928ecf3ee None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:15.098165 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa89a80-16a9-49f7-bda3-ab3928ecf3ee None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:15.098627 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa89a80-16a9-49f7-bda3-ab3928ecf3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:15.103532 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-75b61dd3-82ba-48d1-bc47-bf04e645b50e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:47:15.103532 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:15.103532 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:15.103532 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:15.103532 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:15.103532 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:15.103532 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:15.103532 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:15.103532 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:15.103532 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:15.103532 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:15.103532 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:15.103532 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:47:15.103532 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:47:15.103532 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:47:15.103532 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:47:15.104394 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:47:15.104394 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:47:15.104394 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:47:15.104394 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:47:15.104394 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:15.104394 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:15.104394 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:15.104394 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:15.104394 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:15.104394 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:15.104394 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:15.104394 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:15.104394 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:15.104394 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:15.104394 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:15.104394 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:15.105157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:15.105157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:15.105157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:15.105157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:15.105157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:47:15.105157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:47:15.105157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:47:15.105157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:47:15.105157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:47:15.105157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:47:15.105157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:47:15.105157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:47:15.105157 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 14/27] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:47:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:47:15.112415 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b61dd3-82ba-48d1-bc47-bf04e645b50e None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:15.117790 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:15.141580 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-baa89a80-16a9-49f7-bda3-ab3928ecf3ee None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:15.142849 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-baa89a80-16a9-49f7-bda3-ab3928ecf3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:15.268298 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 14/28] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:15.269848 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa89a80-16a9-49f7-bda3-ab3928ecf3ee None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:15.301195 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b61dd3-82ba-48d1-bc47-bf04e645b50e None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:15.372797 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00192b0b-ea12-4e61-8b3f-cef86986c52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:15.373419 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00192b0b-ea12-4e61-8b3f-cef86986c52d None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:15.373743 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00192b0b-ea12-4e61-8b3f-cef86986c52d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:15.374033 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00192b0b-ea12-4e61-8b3f-cef86986c52d None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:15.374719 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00192b0b-ea12-4e61-8b3f-cef86986c52d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:15.375026 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00192b0b-ea12-4e61-8b3f-cef86986c52d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:15.376434 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00192b0b-ea12-4e61-8b3f-cef86986c52d None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:15.390551 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 15/29] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:47:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:15.392588 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b90b9-65fa-4972-a4b9-0c00ea2a7085 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:15.393102 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b90b9-65fa-4972-a4b9-0c00ea2a7085 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:15.393573 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b90b9-65fa-4972-a4b9-0c00ea2a7085 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:15.395709 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 16/30] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:15.397747 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b90b9-65fa-4972-a4b9-0c00ea2a7085 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:15.489559 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:15.509719 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa89a80-16a9-49f7-bda3-ab3928ecf3ee None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:15.520853 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:15.533669 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b90b9-65fa-4972-a4b9-0c00ea2a7085 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:15.573049 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6627512-02c8-4423-a8b9-ff10a2a7e96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:15.573689 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6627512-02c8-4423-a8b9-ff10a2a7e96c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:15.573973 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6627512-02c8-4423-a8b9-ff10a2a7e96c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:15.574446 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6627512-02c8-4423-a8b9-ff10a2a7e96c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:15.593158 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6627512-02c8-4423-a8b9-ff10a2a7e96c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:15.594650 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6627512-02c8-4423-a8b9-ff10a2a7e96c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:15.601220 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b04d0-fb8f-4028-bbce-e349a0f20a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:15.601856 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b04d0-fb8f-4028-bbce-e349a0f20a9a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:15.602574 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b04d0-fb8f-4028-bbce-e349a0f20a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:15.603623 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b04d0-fb8f-4028-bbce-e349a0f20a9a None admin] PATH_INFO: `/v3/users/644f0b1b120742d2aca15aeeaa76abcc` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:15.606099 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b04d0-fb8f-4028-bbce-e349a0f20a9a None admin] RBAC: Authorizing `identity:get_user(user_id=644f0b1b120742d2aca15aeeaa76abcc)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:15.610323 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b04d0-fb8f-4028-bbce-e349a0f20a9a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:15.612134 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 17/31] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:47:15 2022] GET /identity/v3/users/644f0b1b120742d2aca15aeeaa76abcc => generated 247 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:15.613738 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb36e32-2c99-41e6-a36c-db1a57fd40c3 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:15.614286 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb36e32-2c99-41e6-a36c-db1a57fd40c3 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:15.616194 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb36e32-2c99-41e6-a36c-db1a57fd40c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:15.629244 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-a6627512-02c8-4423-a8b9-ff10a2a7e96c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:47:15.629244 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:15.629244 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:15.629244 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:15.629244 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:15.629244 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:15.629244 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:15.629244 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:15.629244 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:15.629244 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:15.629244 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:15.629244 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:15.629244 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:47:15.629244 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:47:15.629244 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:47:15.629244 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:47:15.630374 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:15.630374 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:15.630374 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:15.630374 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:15.630374 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:15.630374 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:15.630374 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:15.630374 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:15.630374 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:15.630374 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:15.630374 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:15.630374 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:15.630374 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:15.630374 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:15.630374 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:15.630374 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:15.630374 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:47:15.631388 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:47:15.631388 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:47:15.631388 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:47:15.633330 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 15/32] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:47:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:47:15.634327 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6627512-02c8-4423-a8b9-ff10a2a7e96c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:15.641603 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-5eb36e32-2c99-41e6-a36c-db1a57fd40c3 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:15.642932 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5eb36e32-2c99-41e6-a36c-db1a57fd40c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:15.716031 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 18/33] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:15.716957 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb36e32-2c99-41e6-a36c-db1a57fd40c3 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:15.763790 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6627512-02c8-4423-a8b9-ff10a2a7e96c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:15.792584 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:15.822889 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca10257-184e-4aa3-84c7-78d8569f1a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:15.823519 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca10257-184e-4aa3-84c7-78d8569f1a43 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:15.823800 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca10257-184e-4aa3-84c7-78d8569f1a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:15.824144 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca10257-184e-4aa3-84c7-78d8569f1a43 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:15.828756 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca10257-184e-4aa3-84c7-78d8569f1a43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:15.831968 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca10257-184e-4aa3-84c7-78d8569f1a43 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:15.836959 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 16/34] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:47:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:15.837797 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca10257-184e-4aa3-84c7-78d8569f1a43 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:15.867427 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb36e32-2c99-41e6-a36c-db1a57fd40c3 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:15.899322 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:15.958602 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767216e0-ab30-4393-b258-40a63d463a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:15.959726 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767216e0-ab30-4393-b258-40a63d463a59 None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:15.960218 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767216e0-ab30-4393-b258-40a63d463a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:15.960729 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767216e0-ab30-4393-b258-40a63d463a59 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:15.961638 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767216e0-ab30-4393-b258-40a63d463a59 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:15.963242 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767216e0-ab30-4393-b258-40a63d463a59 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:15.983476 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca10257-184e-4aa3-84c7-78d8569f1a43 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:16.021216 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f980184-9ef3-48f6-9195-7f4da6b67b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:16.021711 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f980184-9ef3-48f6-9195-7f4da6b67b9b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:16.021991 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f980184-9ef3-48f6-9195-7f4da6b67b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:16.022433 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f980184-9ef3-48f6-9195-7f4da6b67b9b None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:16.023077 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f980184-9ef3-48f6-9195-7f4da6b67b9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:16.023379 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f980184-9ef3-48f6-9195-7f4da6b67b9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:16.024777 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f980184-9ef3-48f6-9195-7f4da6b67b9b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:16.038132 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 17/35] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:47:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:16.039343 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f980184-9ef3-48f6-9195-7f4da6b67b9b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:16.130744 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:16.145861 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f980184-9ef3-48f6-9195-7f4da6b67b9b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:16.180842 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796b592-b324-4ee8-bf6f-823253394b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:16.181496 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796b592-b324-4ee8-bf6f-823253394b80 None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:16.181777 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796b592-b324-4ee8-bf6f-823253394b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:16.182053 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796b592-b324-4ee8-bf6f-823253394b80 None admin] PATH_INFO: `/v3/domains/default/users/644f0b1b120742d2aca15aeeaa76abcc/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:16.184282 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0796b592-b324-4ee8-bf6f-823253394b80 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=644f0b1b120742d2aca15aeeaa76abcc, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:16.185785 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0796b592-b324-4ee8-bf6f-823253394b80 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:16.193901 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 19/36] 10.222.0.27 () {60 vars in 1131 bytes} [Sun Sep 4 00:47:15 2022] POST /identity/v3/roles => generated 219 bytes in 477 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:47:16.194882 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767216e0-ab30-4393-b258-40a63d463a59 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:16.322171 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767216e0-ab30-4393-b258-40a63d463a59 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:16.350183 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:16.362430 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e53e0-3d47-440d-8909-c232601bf539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:16.363042 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e53e0-3d47-440d-8909-c232601bf539 None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:16.363325 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e53e0-3d47-440d-8909-c232601bf539 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:16.363599 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e53e0-3d47-440d-8909-c232601bf539 None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:16.364255 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6e53e0-3d47-440d-8909-c232601bf539 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:16.365936 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6e53e0-3d47-440d-8909-c232601bf539 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:16.408996 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 18/37] 10.222.0.27 () {58 vars in 1361 bytes} [Sun Sep 4 00:47:16 2022] PUT /identity/v3/domains/default/users/644f0b1b120742d2aca15aeeaa76abcc/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 00:47:16.410211 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796b592-b324-4ee8-bf6f-823253394b80 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:16.457021 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 20/38] 10.222.0.27 () {60 vars in 1141 bytes} [Sun Sep 4 00:47:16 2022] POST /identity/v3/projects => generated 301 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:47:16.459677 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83333ad8-c45e-4f36-9f52-f7159e312c1a None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:16.460167 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83333ad8-c45e-4f36-9f52-f7159e312c1a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:16.460481 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83333ad8-c45e-4f36-9f52-f7159e312c1a None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:16.462246 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 21/39] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:16.463192 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d8aa7-2cd2-49e2-aff5-51bba48defe6 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:16.463496 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d8aa7-2cd2-49e2-aff5-51bba48defe6 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:16.463862 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d8aa7-2cd2-49e2-aff5-51bba48defe6 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:16.465158 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 22/40] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:16.468095 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf21502-8fbb-434d-a7d0-32b0c894467a None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:16.468095 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf21502-8fbb-434d-a7d0-32b0c894467a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:16.468302 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf21502-8fbb-434d-a7d0-32b0c894467a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:16.496407 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-dcf21502-8fbb-434d-a7d0-32b0c894467a None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:16.498489 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-dcf21502-8fbb-434d-a7d0-32b0c894467a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:16.573692 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796b592-b324-4ee8-bf6f-823253394b80 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:16.618915 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013074e4-11ce-4a0b-835c-22b2e3297ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:16.619551 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013074e4-11ce-4a0b-835c-22b2e3297ee6 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:16.619835 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013074e4-11ce-4a0b-835c-22b2e3297ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:16.620110 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013074e4-11ce-4a0b-835c-22b2e3297ee6 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:16.621559 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013074e4-11ce-4a0b-835c-22b2e3297ee6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:16.623227 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013074e4-11ce-4a0b-835c-22b2e3297ee6 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:16.688610 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 23/41] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 00:47:16.689863 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0fde3-5f3d-46dd-abfc-291c9097fb84 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:16.690431 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0fde3-5f3d-46dd-abfc-291c9097fb84 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:16.690719 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0fde3-5f3d-46dd-abfc-291c9097fb84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:16.712001 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-d6e0fde3-5f3d-46dd-abfc-291c9097fb84 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:16.716714 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 19/42] 10.222.0.27 () {60 vars in 1131 bytes} [Sun Sep 4 00:47:16 2022] POST /identity/v3/roles => generated 225 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:47:16.716855 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d6e0fde3-5f3d-46dd-abfc-291c9097fb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:16.805202 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 24/43] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:16.985762 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013074e4-11ce-4a0b-835c-22b2e3297ee6 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:16.996165 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:17.081803 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0fde3-5f3d-46dd-abfc-291c9097fb84 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:17.090544 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013074e4-11ce-4a0b-835c-22b2e3297ee6 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:17.118128 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:17.126931 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ecfa27-207a-43b8-b451-5f939d13ca4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:17.127590 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ecfa27-207a-43b8-b451-5f939d13ca4c None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:17.127908 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ecfa27-207a-43b8-b451-5f939d13ca4c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:17.128229 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ecfa27-207a-43b8-b451-5f939d13ca4c None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:17.129004 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ecfa27-207a-43b8-b451-5f939d13ca4c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:17.130387 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ecfa27-207a-43b8-b451-5f939d13ca4c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:17.185807 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0fde3-5f3d-46dd-abfc-291c9097fb84 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:17.203732 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 20/44] 10.222.0.27 () {60 vars in 1131 bytes} [Sun Sep 4 00:47:16 2022] POST /identity/v3/roles => generated 223 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:47:17.230420 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fae1fc-34d4-4be5-8138-f7fe89c6cb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:17.231025 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fae1fc-34d4-4be5-8138-f7fe89c6cb2f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:17.231317 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fae1fc-34d4-4be5-8138-f7fe89c6cb2f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:17.231604 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fae1fc-34d4-4be5-8138-f7fe89c6cb2f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:17.232874 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fae1fc-34d4-4be5-8138-f7fe89c6cb2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:17.234398 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fae1fc-34d4-4be5-8138-f7fe89c6cb2f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:17.236410 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 25/45] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:47:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:47:17.242438 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ecfa27-207a-43b8-b451-5f939d13ca4c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:17.322068 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:17.360574 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ecfa27-207a-43b8-b451-5f939d13ca4c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:17.397462 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fea1ccb-bd9c-443d-8a41-ce7dd8121a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:17.398425 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea1ccb-bd9c-443d-8a41-ce7dd8121a03 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:17.398930 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea1ccb-bd9c-443d-8a41-ce7dd8121a03 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:17.399405 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea1ccb-bd9c-443d-8a41-ce7dd8121a03 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:17.400071 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fea1ccb-bd9c-443d-8a41-ce7dd8121a03 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:17.401544 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fea1ccb-bd9c-443d-8a41-ce7dd8121a03 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:17.488614 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 21/46] 10.222.0.27 () {60 vars in 1141 bytes} [Sun Sep 4 00:47:17 2022] POST /identity/v3/projects => generated 312 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:47:18.378382 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e162483-d131-44b1-a17d-3ebc655f039b None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:18.378886 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e162483-d131-44b1-a17d-3ebc655f039b None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:18.378964 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e162483-d131-44b1-a17d-3ebc655f039b None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:18.380290 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 26/47] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:18.386007 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d4317-c2e2-492a-889f-08f9f35d0f04 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:18.386422 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d4317-c2e2-492a-889f-08f9f35d0f04 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:18.386579 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d4317-c2e2-492a-889f-08f9f35d0f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:18.508657 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-044d4317-c2e2-492a-889f-08f9f35d0f04 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:18.510205 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-044d4317-c2e2-492a-889f-08f9f35d0f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:18.578613 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 22/48] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:18.855086 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e162483-d131-44b1-a17d-3ebc655f039b None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:18.956925 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e162483-d131-44b1-a17d-3ebc655f039b None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:18.959853 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:18.991170 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ecdbce-df65-457c-8890-cfd33b39717a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:18.991794 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ecdbce-df65-457c-8890-cfd33b39717a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:18.992074 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ecdbce-df65-457c-8890-cfd33b39717a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:18.992343 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ecdbce-df65-457c-8890-cfd33b39717a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:19.002749 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ecdbce-df65-457c-8890-cfd33b39717a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:47:19.008432 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-b3ecdbce-df65-457c-8890-cfd33b39717a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:47:19.008432 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:19.008432 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:19.008432 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:19.008432 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:19.008432 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:19.008432 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:19.008432 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:19.008432 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:19.008432 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:19.008432 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:19.008432 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:19.008432 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:47:19.008432 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:47:19.008432 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:47:19.008432 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:47:19.009457 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:47:19.009457 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:47:19.009457 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:47:19.009457 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:47:19.009457 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:19.009457 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:19.009457 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:19.009457 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:19.009457 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:19.009457 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:19.009457 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:19.009457 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:19.009457 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:19.009457 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:19.009457 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:19.009457 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:19.010318 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:19.010318 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:19.010318 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:19.010318 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:19.010318 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:47:19.010318 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:47:19.010318 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:47:19.010318 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:47:19.010318 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:47:19.010318 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:47:19.010318 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:47:19.010318 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:47:19.010318 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 27/49] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:47:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:47:19.015369 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d4317-c2e2-492a-889f-08f9f35d0f04 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:19.064672 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:19.117549 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d4317-c2e2-492a-889f-08f9f35d0f04 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:19.151295 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9754f-fa68-44f1-8666-9a5d55ccfa5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:19.151901 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9754f-fa68-44f1-8666-9a5d55ccfa5e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:19.152182 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9754f-fa68-44f1-8666-9a5d55ccfa5e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:19.152471 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9754f-fa68-44f1-8666-9a5d55ccfa5e None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:19.153142 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf9754f-fa68-44f1-8666-9a5d55ccfa5e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:19.153443 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf9754f-fa68-44f1-8666-9a5d55ccfa5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:19.154828 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf9754f-fa68-44f1-8666-9a5d55ccfa5e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:19.166957 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 23/50] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:47:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:19.172794 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ecdbce-df65-457c-8890-cfd33b39717a None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:19.275257 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ecdbce-df65-457c-8890-cfd33b39717a None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:19.309435 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c0e76c-9676-4fc6-89df-812823790e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:19.310070 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0e76c-9676-4fc6-89df-812823790e52 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:19.310356 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0e76c-9676-4fc6-89df-812823790e52 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:19.310638 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0e76c-9676-4fc6-89df-812823790e52 None admin] PATH_INFO: `/v3/users/644f0b1b120742d2aca15aeeaa76abcc` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:19.311763 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c0e76c-9676-4fc6-89df-812823790e52 None admin] RBAC: Authorizing `identity:get_user(user_id=644f0b1b120742d2aca15aeeaa76abcc)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:19.313264 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c0e76c-9676-4fc6-89df-812823790e52 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:19.314854 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 28/51] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:47:19 2022] GET /identity/v3/users/644f0b1b120742d2aca15aeeaa76abcc => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:19.321100 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9754f-fa68-44f1-8666-9a5d55ccfa5e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:19.399302 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156bf44-c14c-4182-8424-bcdd6433943a None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:19.399789 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156bf44-c14c-4182-8424-bcdd6433943a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:19.399789 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156bf44-c14c-4182-8424-bcdd6433943a None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:19.401157 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 29/52] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 00:47:19.405963 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf0620-e485-421c-862c-3a10d084af99 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:19.406251 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf0620-e485-421c-862c-3a10d084af99 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:19.406532 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf0620-e485-421c-862c-3a10d084af99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:19.420735 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9754f-fa68-44f1-8666-9a5d55ccfa5e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:19.425007 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-72bf0620-e485-421c-862c-3a10d084af99 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:19.426215 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-72bf0620-e485-421c-862c-3a10d084af99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:19.431822 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:19.453846 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:19.455481 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b753a-5593-4069-acdf-2e02c1bf1bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:19.456130 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b753a-5593-4069-acdf-2e02c1bf1bda None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:19.456415 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b753a-5593-4069-acdf-2e02c1bf1bda None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:19.456712 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b753a-5593-4069-acdf-2e02c1bf1bda None admin] PATH_INFO: `/v3/domains/default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:19.457837 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7b753a-5593-4069-acdf-2e02c1bf1bda None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:19.459593 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7b753a-5593-4069-acdf-2e02c1bf1bda None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:19.461231 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 24/53] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:47:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:19.467258 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b753a-5593-4069-acdf-2e02c1bf1bda None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:19.496443 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 30/54] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 00:47:19.569157 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b753a-5593-4069-acdf-2e02c1bf1bda None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:19.602347 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc2ad9-cadb-4ce7-8085-f4a10d143436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:19.602955 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc2ad9-cadb-4ce7-8085-f4a10d143436 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:19.603242 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc2ad9-cadb-4ce7-8085-f4a10d143436 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:19.603520 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc2ad9-cadb-4ce7-8085-f4a10d143436 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:19.604247 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bc2ad9-cadb-4ce7-8085-f4a10d143436 None admin] RBAC: Adding query filter params (role.id=c8a59d34ee2f4cefa6940cac778260c7, scope.domain.id=default, user.id=644f0b1b120742d2aca15aeeaa76abcc) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:19.604586 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bc2ad9-cadb-4ce7-8085-f4a10d143436 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:19.606035 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bc2ad9-cadb-4ce7-8085-f4a10d143436 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:19.613788 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 25/55] 10.222.0.27 () {56 vars in 1322 bytes} [Sun Sep 4 00:47:19 2022] GET /identity/v3/role_assignments?user.id=644f0b1b120742d2aca15aeeaa76abcc&scope.domain.id=default&role.id=c8a59d34ee2f4cefa6940cac778260c7 => generated 530 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:47:19.767593 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf0620-e485-421c-862c-3a10d084af99 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:19.867371 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf0620-e485-421c-862c-3a10d084af99 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:19.900945 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5536d-ab5c-4277-9c5f-4c78cc696251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:19.901577 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5536d-ab5c-4277-9c5f-4c78cc696251 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:19.901864 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5536d-ab5c-4277-9c5f-4c78cc696251 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:19.902146 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5536d-ab5c-4277-9c5f-4c78cc696251 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:19.903246 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf5536d-ab5c-4277-9c5f-4c78cc696251 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:19.904801 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf5536d-ab5c-4277-9c5f-4c78cc696251 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:19.906359 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 31/56] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:47:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:19.912089 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc2ad9-cadb-4ce7-8085-f4a10d143436 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:19.955553 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:20.012043 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc2ad9-cadb-4ce7-8085-f4a10d143436 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:20.044287 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49299dd7-0df0-44b5-8bb9-2c17b30b2083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:20.044856 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49299dd7-0df0-44b5-8bb9-2c17b30b2083 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:20.045138 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49299dd7-0df0-44b5-8bb9-2c17b30b2083 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:20.045412 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49299dd7-0df0-44b5-8bb9-2c17b30b2083 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:20.046052 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49299dd7-0df0-44b5-8bb9-2c17b30b2083 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:20.047478 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49299dd7-0df0-44b5-8bb9-2c17b30b2083 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:20.128083 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 26/57] 10.222.0.27 () {60 vars in 1140 bytes} [Sun Sep 4 00:47:19 2022] POST /identity/v3/projects => generated 298 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:47:22.015797 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574377c-ae5f-4200-af47-357015578f05 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:22.016309 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574377c-ae5f-4200-af47-357015578f05 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:22.016391 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574377c-ae5f-4200-af47-357015578f05 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:22.017705 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 32/58] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:22.022760 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4997adf-7a52-4b7a-a666-67e73c5a869c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:22.023156 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4997adf-7a52-4b7a-a666-67e73c5a869c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:22.023305 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4997adf-7a52-4b7a-a666-67e73c5a869c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:22.041365 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f4997adf-7a52-4b7a-a666-67e73c5a869c None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:22.042541 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f4997adf-7a52-4b7a-a666-67e73c5a869c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:22.107016 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 27/59] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:22.374520 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7574377c-ae5f-4200-af47-357015578f05 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:22.404474 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:22.473377 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7574377c-ae5f-4200-af47-357015578f05 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:22.506593 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d602e0e-9acf-4fe4-91d7-735f8af8c85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:22.507204 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d602e0e-9acf-4fe4-91d7-735f8af8c85a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:22.507486 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d602e0e-9acf-4fe4-91d7-735f8af8c85a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:22.507758 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d602e0e-9acf-4fe4-91d7-735f8af8c85a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:22.508874 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d602e0e-9acf-4fe4-91d7-735f8af8c85a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:22.510305 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d602e0e-9acf-4fe4-91d7-735f8af8c85a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:22.511832 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 33/60] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:47:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:22.517290 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4997adf-7a52-4b7a-a666-67e73c5a869c None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:22.542041 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:22.615199 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4997adf-7a52-4b7a-a666-67e73c5a869c None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:22.648188 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4e261c-112f-4593-b53d-7e21b483cc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:22.648757 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e261c-112f-4593-b53d-7e21b483cc2d None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:22.649041 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e261c-112f-4593-b53d-7e21b483cc2d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:22.649316 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e261c-112f-4593-b53d-7e21b483cc2d None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:22.649965 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4e261c-112f-4593-b53d-7e21b483cc2d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:22.651407 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4e261c-112f-4593-b53d-7e21b483cc2d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:22.743878 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 28/61] 10.222.0.27 () {60 vars in 1132 bytes} [Sun Sep 4 00:47:22 2022] POST /identity/v3/users => generated 275 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 00:47:25.782068 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5fff5-60e7-4c95-9da3-e7b840042012 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:25.782907 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5fff5-60e7-4c95-9da3-e7b840042012 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:25.783353 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5fff5-60e7-4c95-9da3-e7b840042012 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:25.784866 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 34/62] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:25.791193 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d754df-1b54-4e3a-8146-5dca45f4a99b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:25.791598 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d754df-1b54-4e3a-8146-5dca45f4a99b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:25.791754 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d754df-1b54-4e3a-8146-5dca45f4a99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:25.816992 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3969dd85-1e29-4a86-8b1f-98ade368117e None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:25.817163 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-75d754df-1b54-4e3a-8146-5dca45f4a99b None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:25.817685 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3969dd85-1e29-4a86-8b1f-98ade368117e None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:25.818160 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3969dd85-1e29-4a86-8b1f-98ade368117e None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:25.818879 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-75d754df-1b54-4e3a-8146-5dca45f4a99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:25.820415 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 35/63] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 00:47:25.825780 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07fb28-b0e0-426b-b626-6b1befed7722 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:25.826308 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07fb28-b0e0-426b-b626-6b1befed7722 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:25.826781 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07fb28-b0e0-426b-b626-6b1befed7722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:25.858910 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-3a07fb28-b0e0-426b-b626-6b1befed7722 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:25.860618 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3a07fb28-b0e0-426b-b626-6b1befed7722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:25.950138 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 36/64] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:25.970158 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 29/65] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:26.026038 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4db85-acd9-4207-b02d-b6b8ef76b41d None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:26.026575 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4db85-acd9-4207-b02d-b6b8ef76b41d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:26.027008 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4db85-acd9-4207-b02d-b6b8ef76b41d None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:26.028553 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 37/66] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:26.033966 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2d4f0-e92a-42ba-8909-db48ac997605 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:26.034203 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2d4f0-e92a-42ba-8909-db48ac997605 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:26.034490 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2d4f0-e92a-42ba-8909-db48ac997605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:26.060737 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-0be2d4f0-e92a-42ba-8909-db48ac997605 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:26.061948 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0be2d4f0-e92a-42ba-8909-db48ac997605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:26.065584 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:26.145368 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 30/67] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:26.322836 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4db85-acd9-4207-b02d-b6b8ef76b41d None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:26.325171 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be2d4f0-e92a-42ba-8909-db48ac997605 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:26.364724 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:26.455857 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be2d4f0-e92a-42ba-8909-db48ac997605 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:26.465447 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4db85-acd9-4207-b02d-b6b8ef76b41d None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:26.497923 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddecf9fb-05e8-4538-b13d-e98959db01d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:26.506406 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddecf9fb-05e8-4538-b13d-e98959db01d4 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:26.506406 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddecf9fb-05e8-4538-b13d-e98959db01d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:26.506406 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddecf9fb-05e8-4538-b13d-e98959db01d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:26.506694 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e46af-0418-4865-a41c-03affc866383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:26.506694 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e46af-0418-4865-a41c-03affc866383 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:26.506694 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e46af-0418-4865-a41c-03affc866383 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:26.506694 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e46af-0418-4865-a41c-03affc866383 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:26.509760 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:26.513166 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddecf9fb-05e8-4538-b13d-e98959db01d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:47:26.516533 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e46af-0418-4865-a41c-03affc866383 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:47:26.520888 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-ddecf9fb-05e8-4538-b13d-e98959db01d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:47:26.520888 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:26.520888 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:26.520888 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:26.520888 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:26.520888 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:26.520888 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:26.520888 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:26.520888 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:26.520888 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:26.520888 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:26.520888 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:26.520888 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:47:26.520888 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:47:26.520888 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:47:26.520888 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:47:26.531918 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:47:26.531918 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:47:26.531918 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:47:26.531918 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:47:26.531918 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:26.531918 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:26.531918 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:26.531918 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:26.531918 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:26.531918 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:26.531918 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:26.531918 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:26.531918 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:26.531918 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:26.531918 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:26.531918 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:26.533115 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:26.533115 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:26.533115 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:26.533115 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:26.533115 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:47:26.533115 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:47:26.533115 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:47:26.533115 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:47:26.533115 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:47:26.533115 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:47:26.533115 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:47:26.533115 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:47:26.533675 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-c07e46af-0418-4865-a41c-03affc866383 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:47:26.533675 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:26.533675 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:26.533675 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:26.533675 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:26.533675 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:26.533675 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:26.533675 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:26.533675 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:26.533675 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:26.533675 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:26.533675 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:26.533675 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:47:26.533675 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:47:26.533675 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:47:26.533675 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:47:26.534415 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:47:26.534415 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:47:26.534415 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:47:26.534415 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:47:26.534415 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:26.534415 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:26.534415 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:26.534415 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:26.534415 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:26.534415 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:26.534415 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:26.534415 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:26.534415 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:26.534415 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:26.534415 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:26.534415 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:26.535452 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:26.535452 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:26.535452 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:26.535452 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:26.535452 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:47:26.535452 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:47:26.535452 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:47:26.535452 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:47:26.535452 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:47:26.535452 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:47:26.535452 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:47:26.535452 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:47:26.536196 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 31/68] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:47:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:47:26.536273 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 38/69] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:47:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:47:26.536348 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762958ca-bfcf-4bf3-82b1-7e1758f4a7e1 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:26.536348 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762958ca-bfcf-4bf3-82b1-7e1758f4a7e1 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:26.536348 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762958ca-bfcf-4bf3-82b1-7e1758f4a7e1 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:26.536582 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0da9c4-2795-4b6e-aa06-42e044fbd5a5 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:26.536582 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0da9c4-2795-4b6e-aa06-42e044fbd5a5 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:26.536582 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0da9c4-2795-4b6e-aa06-42e044fbd5a5 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:26.536778 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 32/70] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:26.536860 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 39/71] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:26.536936 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762958ca-bfcf-4bf3-82b1-7e1758f4a7e1 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:26.537011 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0da9c4-2795-4b6e-aa06-42e044fbd5a5 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:26.622579 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:26.631493 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762958ca-bfcf-4bf3-82b1-7e1758f4a7e1 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:26.635622 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0da9c4-2795-4b6e-aa06-42e044fbd5a5 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:26.665268 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b2a13-2702-4d3e-9249-2ba5a63edbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:26.665887 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b2a13-2702-4d3e-9249-2ba5a63edbed None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:26.666168 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b2a13-2702-4d3e-9249-2ba5a63edbed None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:26.666442 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b2a13-2702-4d3e-9249-2ba5a63edbed None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:26.670186 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a28c5-34e1-4e35-a45a-57d5097ae315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:26.670791 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a28c5-34e1-4e35-a45a-57d5097ae315 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:26.671079 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a28c5-34e1-4e35-a45a-57d5097ae315 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:26.671362 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a28c5-34e1-4e35-a45a-57d5097ae315 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:26.672026 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4a28c5-34e1-4e35-a45a-57d5097ae315 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:26.672333 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4a28c5-34e1-4e35-a45a-57d5097ae315 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:26.673784 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4a28c5-34e1-4e35-a45a-57d5097ae315 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:26.676862 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7b2a13-2702-4d3e-9249-2ba5a63edbed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:47:26.681785 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-ce7b2a13-2702-4d3e-9249-2ba5a63edbed None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 00:47:26.681785 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:26.681785 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:26.681785 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:26.681785 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:26.681785 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:26.681785 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:26.681785 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:26.681785 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:26.681785 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:26.681785 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:26.681785 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:26.681785 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:47:26.681785 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:47:26.681785 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:47:26.681785 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:47:26.682687 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:47:26.682687 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:47:26.682687 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:47:26.682687 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:47:26.682687 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:26.682687 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:26.682687 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:26.682687 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:26.682687 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:26.682687 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:26.682687 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:26.682687 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:26.682687 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:26.682687 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:26.682687 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:26.682687 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:26.683437 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:26.683437 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:26.683437 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:26.683437 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:26.683437 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:47:26.683437 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:47:26.683437 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:47:26.683437 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:47:26.683437 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:47:26.683437 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:47:26.683437 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 00:47:26.683437 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:47:26.684012 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 40/72] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:47:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:47:26.684012 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea208bf-0616-4c22-9677-47337b1eee26 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:26.684012 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea208bf-0616-4c22-9677-47337b1eee26 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:26.684012 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea208bf-0616-4c22-9677-47337b1eee26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:26.688597 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 33/73] 10.222.0.27 () {56 vars in 1114 bytes} [Sun Sep 4 00:47:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:47:26.689665 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b2a13-2702-4d3e-9249-2ba5a63edbed None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:26.705454 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-eea208bf-0616-4c22-9677-47337b1eee26 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:26.706662 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-eea208bf-0616-4c22-9677-47337b1eee26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:26.749952 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:26.772754 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 41/74] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:26.774170 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c0063-c8a7-43eb-a16c-c09701c2d677 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:26.774457 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c0063-c8a7-43eb-a16c-c09701c2d677 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:26.774734 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c0063-c8a7-43eb-a16c-c09701c2d677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:26.790558 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b2a13-2702-4d3e-9249-2ba5a63edbed None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:26.800405 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-9c1c0063-c8a7-43eb-a16c-c09701c2d677 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:26.801631 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-9c1c0063-c8a7-43eb-a16c-c09701c2d677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:26.830424 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2c5af-3f46-4135-bf76-52b28ddf0342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:26.831534 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2c5af-3f46-4135-bf76-52b28ddf0342 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:26.831818 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2c5af-3f46-4135-bf76-52b28ddf0342 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:26.832093 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2c5af-3f46-4135-bf76-52b28ddf0342 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:26.832785 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2c5af-3f46-4135-bf76-52b28ddf0342 None admin] RBAC: Adding query filter params (name=member) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:26.833090 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2c5af-3f46-4135-bf76-52b28ddf0342 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:26.835293 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2c5af-3f46-4135-bf76-52b28ddf0342 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:26.845345 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 34/75] 10.222.0.27 () {56 vars in 1101 bytes} [Sun Sep 4 00:47:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:26.845984 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2c5af-3f46-4135-bf76-52b28ddf0342 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:26.876962 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 42/76] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:26.878040 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c0063-c8a7-43eb-a16c-c09701c2d677 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:26.892805 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:26.985633 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2c5af-3f46-4135-bf76-52b28ddf0342 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:26.994314 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c0063-c8a7-43eb-a16c-c09701c2d677 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:27.018345 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:27.031310 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a2d811-465c-4b15-93eb-3884f8280048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:27.032119 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2d811-465c-4b15-93eb-3884f8280048 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:27.032404 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2d811-465c-4b15-93eb-3884f8280048 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:27.032742 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2d811-465c-4b15-93eb-3884f8280048 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:27.038324 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a2d811-465c-4b15-93eb-3884f8280048 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:27.038324 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a2d811-465c-4b15-93eb-3884f8280048 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:27.038483 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f46f5c-1cf4-46b6-9ec7-7990b1273477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:27.038566 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a2d811-465c-4b15-93eb-3884f8280048 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:27.038633 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f46f5c-1cf4-46b6-9ec7-7990b1273477 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:27.038633 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f46f5c-1cf4-46b6-9ec7-7990b1273477 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:27.038633 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f46f5c-1cf4-46b6-9ec7-7990b1273477 None admin] PATH_INFO: `/v3/users/644f0b1b120742d2aca15aeeaa76abcc` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:27.039164 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f46f5c-1cf4-46b6-9ec7-7990b1273477 None admin] RBAC: Authorizing `identity:get_user(user_id=644f0b1b120742d2aca15aeeaa76abcc)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:27.040930 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f46f5c-1cf4-46b6-9ec7-7990b1273477 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:27.048333 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 43/77] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:47:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:27.048333 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a2d811-465c-4b15-93eb-3884f8280048 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:27.048765 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 35/78] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:47:26 2022] GET /identity/v3/users/644f0b1b120742d2aca15aeeaa76abcc => generated 247 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:27.049616 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f46f5c-1cf4-46b6-9ec7-7990b1273477 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:27.129520 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:27.160751 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a2d811-465c-4b15-93eb-3884f8280048 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:27.163302 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f46f5c-1cf4-46b6-9ec7-7990b1273477 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:27.195128 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec23beb5-47a4-4712-8161-af1cd9a7b56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:27.195761 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23beb5-47a4-4712-8161-af1cd9a7b56e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:27.196049 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23beb5-47a4-4712-8161-af1cd9a7b56e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:27.196329 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23beb5-47a4-4712-8161-af1cd9a7b56e None admin] PATH_INFO: `/v3/users/43855050fbcd47d3b5ded0e8ac79e20d` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:27.198210 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f78ca8f-10f6-48c9-a550-caa80dc67456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:27.198839 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78ca8f-10f6-48c9-a550-caa80dc67456 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:27.199128 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78ca8f-10f6-48c9-a550-caa80dc67456 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:27.199407 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78ca8f-10f6-48c9-a550-caa80dc67456 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:27.210540 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f78ca8f-10f6-48c9-a550-caa80dc67456 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:47:27.211369 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec23beb5-47a4-4712-8161-af1cd9a7b56e None admin] RBAC: Authorizing `identity:get_user(user_id=43855050fbcd47d3b5ded0e8ac79e20d)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:27.212867 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec23beb5-47a4-4712-8161-af1cd9a7b56e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:27.214493 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 44/79] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:47:27 2022] GET /identity/v3/users/43855050fbcd47d3b5ded0e8ac79e20d => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:27.215269 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec23beb5-47a4-4712-8161-af1cd9a7b56e None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:27.215429 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-1f78ca8f-10f6-48c9-a550-caa80dc67456 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:47:27.215429 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:27.215429 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:27.215429 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:27.215429 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:27.215429 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:27.215429 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:27.215429 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:27.215429 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:27.215429 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:27.215429 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:27.215429 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:27.215429 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:47:27.215429 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:47:27.215429 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:47:27.215429 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:47:27.216345 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:47:27.216345 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:47:27.216345 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:47:27.216345 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:47:27.216345 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:27.216345 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:27.216345 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:27.216345 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:27.216345 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:27.216345 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:27.216345 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:27.216345 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:27.216345 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:27.216345 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:27.216345 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:27.216345 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:27.221026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:27.221026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:27.221026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:27.221026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:27.221026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:47:27.221026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:47:27.221026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:47:27.221026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:47:27.221026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:47:27.221026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:47:27.221026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:47:27.221026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:47:27.221026 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 36/80] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:47:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:47:27.221026 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f78ca8f-10f6-48c9-a550-caa80dc67456 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:27.269083 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:27.318936 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f78ca8f-10f6-48c9-a550-caa80dc67456 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:27.319466 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec23beb5-47a4-4712-8161-af1cd9a7b56e None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:27.347716 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:27.353272 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685fec8e-2ae7-4398-9fef-e3ad06da65b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:27.353890 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fec8e-2ae7-4398-9fef-e3ad06da65b4 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:27.354174 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fec8e-2ae7-4398-9fef-e3ad06da65b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:27.358674 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fec8e-2ae7-4398-9fef-e3ad06da65b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:27.358768 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da592413-c774-4beb-930f-979b56d73c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:27.358768 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da592413-c774-4beb-930f-979b56d73c0e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:27.358768 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da592413-c774-4beb-930f-979b56d73c0e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:27.358768 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da592413-c774-4beb-930f-979b56d73c0e None admin] PATH_INFO: `/v3/users/43855050fbcd47d3b5ded0e8ac79e20d` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:27.359016 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da592413-c774-4beb-930f-979b56d73c0e None admin] RBAC: Authorizing `identity:get_user(user_id=43855050fbcd47d3b5ded0e8ac79e20d)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:27.359016 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da592413-c774-4beb-930f-979b56d73c0e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:27.359939 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 37/81] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:47:27 2022] GET /identity/v3/users/43855050fbcd47d3b5ded0e8ac79e20d => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:27.360735 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da592413-c774-4beb-930f-979b56d73c0e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:27.422617 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685fec8e-2ae7-4398-9fef-e3ad06da65b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:27.424021 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685fec8e-2ae7-4398-9fef-e3ad06da65b4 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:27.460818 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da592413-c774-4beb-930f-979b56d73c0e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:27.477887 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:27.495465 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c67983-cd85-4b68-af65-152db4b7ae0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:27.496088 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c67983-cd85-4b68-af65-152db4b7ae0b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:27.496369 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c67983-cd85-4b68-af65-152db4b7ae0b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:27.496677 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c67983-cd85-4b68-af65-152db4b7ae0b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:27.497808 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c67983-cd85-4b68-af65-152db4b7ae0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:27.499252 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c67983-cd85-4b68-af65-152db4b7ae0b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:27.500844 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 38/82] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:47:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:27.501610 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c67983-cd85-4b68-af65-152db4b7ae0b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:27.524962 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 45/83] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:47:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:47:27.525891 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685fec8e-2ae7-4398-9fef-e3ad06da65b4 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:27.556157 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:27.606914 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c67983-cd85-4b68-af65-152db4b7ae0b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:27.624260 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685fec8e-2ae7-4398-9fef-e3ad06da65b4 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:27.640517 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560bde36-2f2e-45a1-a822-33d2372f2ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:27.641117 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560bde36-2f2e-45a1-a822-33d2372f2ae0 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:27.641398 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560bde36-2f2e-45a1-a822-33d2372f2ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:27.641678 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560bde36-2f2e-45a1-a822-33d2372f2ae0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:27.658284 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e562db03-47df-4b81-a6a3-abcf305ef6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:27.658891 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562db03-47df-4b81-a6a3-abcf305ef6c5 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:27.659173 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562db03-47df-4b81-a6a3-abcf305ef6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:27.659454 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562db03-47df-4b81-a6a3-abcf305ef6c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:27.660117 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e562db03-47df-4b81-a6a3-abcf305ef6c5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:27.660423 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e562db03-47df-4b81-a6a3-abcf305ef6c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:27.661858 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e562db03-47df-4b81-a6a3-abcf305ef6c5 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:27.670190 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 46/84] 10.222.0.27 () {56 vars in 1101 bytes} [Sun Sep 4 00:47:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:27.670986 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e562db03-47df-4b81-a6a3-abcf305ef6c5 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:27.700516 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:27.714434 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560bde36-2f2e-45a1-a822-33d2372f2ae0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:27.715921 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560bde36-2f2e-45a1-a822-33d2372f2ae0 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:27.760470 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:27.768563 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e562db03-47df-4b81-a6a3-abcf305ef6c5 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:27.802203 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba712d20-0ac1-41cc-8ec4-39e2b807968a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:27.803341 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba712d20-0ac1-41cc-8ec4-39e2b807968a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:27.803660 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba712d20-0ac1-41cc-8ec4-39e2b807968a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:27.803946 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba712d20-0ac1-41cc-8ec4-39e2b807968a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:27.814720 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 39/85] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:47:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:47:27.815551 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560bde36-2f2e-45a1-a822-33d2372f2ae0 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:27.873066 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba712d20-0ac1-41cc-8ec4-39e2b807968a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:27.874545 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba712d20-0ac1-41cc-8ec4-39e2b807968a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:27.903357 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:27.918970 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560bde36-2f2e-45a1-a822-33d2372f2ae0 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:27.952981 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68e914-b169-4028-9d30-913c3e2b2d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:27.953513 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68e914-b169-4028-9d30-913c3e2b2d38 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:27.953795 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68e914-b169-4028-9d30-913c3e2b2d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:27.954070 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68e914-b169-4028-9d30-913c3e2b2d38 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:27.954712 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c68e914-b169-4028-9d30-913c3e2b2d38 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:27.956169 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c68e914-b169-4028-9d30-913c3e2b2d38 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:27.971324 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:27.981154 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 47/86] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:47:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:47:27.982076 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba712d20-0ac1-41cc-8ec4-39e2b807968a None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:28.019434 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 40/87] 10.222.0.27 () {60 vars in 1132 bytes} [Sun Sep 4 00:47:27 2022] POST /identity/v3/users => generated 289 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:47:28.020512 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68e914-b169-4028-9d30-913c3e2b2d38 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:28.102443 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba712d20-0ac1-41cc-8ec4-39e2b807968a None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:28.146813 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb3143-350f-4328-b1de-9eb58148ff2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:28.147826 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb3143-350f-4328-b1de-9eb58148ff2f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:28.148287 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb3143-350f-4328-b1de-9eb58148ff2f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:28.148755 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb3143-350f-4328-b1de-9eb58148ff2f None admin] PATH_INFO: `/v3/projects/c8fe5139a2a24ec780814c7f39d5b437` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:28.150368 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edb3143-350f-4328-b1de-9eb58148ff2f None admin] RBAC: Authorizing `identity:get_project(project_id=c8fe5139a2a24ec780814c7f39d5b437)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:28.151934 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edb3143-350f-4328-b1de-9eb58148ff2f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:28.153859 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 48/88] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:47:27 2022] GET /identity/v3/projects/c8fe5139a2a24ec780814c7f39d5b437 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:28.156250 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68e914-b169-4028-9d30-913c3e2b2d38 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:28.157448 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb3143-350f-4328-b1de-9eb58148ff2f None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:28.200800 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d34705-21b2-4e4a-b49e-1a349dc38925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:28.201455 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d34705-21b2-4e4a-b49e-1a349dc38925 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:28.201747 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d34705-21b2-4e4a-b49e-1a349dc38925 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:28.202029 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d34705-21b2-4e4a-b49e-1a349dc38925 None admin] PATH_INFO: `/v3/users/43855050fbcd47d3b5ded0e8ac79e20d` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:28.203466 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d34705-21b2-4e4a-b49e-1a349dc38925 None admin] RBAC: Authorizing `identity:get_user(user_id=43855050fbcd47d3b5ded0e8ac79e20d)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:28.205438 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d34705-21b2-4e4a-b49e-1a349dc38925 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:28.207087 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 41/89] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:47:28 2022] GET /identity/v3/users/43855050fbcd47d3b5ded0e8ac79e20d => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:28.207877 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d34705-21b2-4e4a-b49e-1a349dc38925 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:28.227419 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:28.257564 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:28.319423 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb3143-350f-4328-b1de-9eb58148ff2f None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:28.372174 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d34705-21b2-4e4a-b49e-1a349dc38925 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:28.381691 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e8f1e-0528-4a60-a1de-928072f0ddf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:28.381878 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e8f1e-0528-4a60-a1de-928072f0ddf0 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:28.382177 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e8f1e-0528-4a60-a1de-928072f0ddf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:28.382524 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e8f1e-0528-4a60-a1de-928072f0ddf0 None admin] PATH_INFO: `/v3/projects/c8fe5139a2a24ec780814c7f39d5b437` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:28.384016 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e8f1e-0528-4a60-a1de-928072f0ddf0 None admin] RBAC: Authorizing `identity:get_project(project_id=c8fe5139a2a24ec780814c7f39d5b437)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:28.386159 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e8f1e-0528-4a60-a1de-928072f0ddf0 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:28.387460 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 49/90] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:47:28 2022] GET /identity/v3/projects/c8fe5139a2a24ec780814c7f39d5b437 => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:28.389262 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e8f1e-0528-4a60-a1de-928072f0ddf0 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:28.426457 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4793c-b14d-4b35-b1d5-373f629bd295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:28.428001 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4793c-b14d-4b35-b1d5-373f629bd295 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:28.428802 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4793c-b14d-4b35-b1d5-373f629bd295 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:28.429293 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4793c-b14d-4b35-b1d5-373f629bd295 None admin] PATH_INFO: `/v3/projects/c8fe5139a2a24ec780814c7f39d5b437` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:28.430679 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4793c-b14d-4b35-b1d5-373f629bd295 None admin] RBAC: Authorizing `identity:get_project(project_id=c8fe5139a2a24ec780814c7f39d5b437)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:28.432565 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4793c-b14d-4b35-b1d5-373f629bd295 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:28.435324 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 42/91] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:47:28 2022] GET /identity/v3/projects/c8fe5139a2a24ec780814c7f39d5b437 => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:28.436362 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4793c-b14d-4b35-b1d5-373f629bd295 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:28.514891 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e8f1e-0528-4a60-a1de-928072f0ddf0 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:28.551899 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:28.554495 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:28.560862 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1a590-156f-4d26-bd1e-5b7572b095d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:28.561472 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1a590-156f-4d26-bd1e-5b7572b095d6 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:28.561759 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1a590-156f-4d26-bd1e-5b7572b095d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:28.562036 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1a590-156f-4d26-bd1e-5b7572b095d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:28.566147 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef1a590-156f-4d26-bd1e-5b7572b095d6 None admin] RBAC: Adding query filter params (role.id=c8a59d34ee2f4cefa6940cac778260c7, scope.project.id=c8fe5139a2a24ec780814c7f39d5b437, user.id=644f0b1b120742d2aca15aeeaa76abcc) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:28.566147 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef1a590-156f-4d26-bd1e-5b7572b095d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:28.566147 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef1a590-156f-4d26-bd1e-5b7572b095d6 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:28.570724 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 50/92] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:47:28 2022] GET /identity/v3/role_assignments?user.id=644f0b1b120742d2aca15aeeaa76abcc&scope.project.id=c8fe5139a2a24ec780814c7f39d5b437&role.id=c8a59d34ee2f4cefa6940cac778260c7 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:28.572234 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1a590-156f-4d26-bd1e-5b7572b095d6 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:28.582075 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4793c-b14d-4b35-b1d5-373f629bd295 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:28.633702 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d4fbb-7cb0-4f26-9b67-414ef42c5d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:28.634588 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d4fbb-7cb0-4f26-9b67-414ef42c5d3a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:28.638999 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d4fbb-7cb0-4f26-9b67-414ef42c5d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:28.639514 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d4fbb-7cb0-4f26-9b67-414ef42c5d3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:28.768025 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d4fbb-7cb0-4f26-9b67-414ef42c5d3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:28.773676 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d4fbb-7cb0-4f26-9b67-414ef42c5d3a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:28.839186 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1a590-156f-4d26-bd1e-5b7572b095d6 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:28.881886 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82872abc-15ff-4e2e-aff7-fbc0a956a29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:28.883031 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82872abc-15ff-4e2e-aff7-fbc0a956a29f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:28.887907 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82872abc-15ff-4e2e-aff7-fbc0a956a29f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:28.887907 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82872abc-15ff-4e2e-aff7-fbc0a956a29f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:28.887907 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82872abc-15ff-4e2e-aff7-fbc0a956a29f None admin] RBAC: Adding query filter params (role.id=d1261c99e1d64f9da63adfff739561e8, scope.project.id=c8fe5139a2a24ec780814c7f39d5b437, user.id=43855050fbcd47d3b5ded0e8ac79e20d) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:28.887907 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82872abc-15ff-4e2e-aff7-fbc0a956a29f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:28.887907 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82872abc-15ff-4e2e-aff7-fbc0a956a29f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:28.894836 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 51/93] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:47:28 2022] GET /identity/v3/role_assignments?user.id=43855050fbcd47d3b5ded0e8ac79e20d&scope.project.id=c8fe5139a2a24ec780814c7f39d5b437&role.id=d1261c99e1d64f9da63adfff739561e8 => generated 259 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:28.896845 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82872abc-15ff-4e2e-aff7-fbc0a956a29f None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:28.945573 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:28.995876 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 43/94] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:47:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:47:29.003377 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d4fbb-7cb0-4f26-9b67-414ef42c5d3a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:29.080365 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:29.136341 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d4fbb-7cb0-4f26-9b67-414ef42c5d3a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:29.167508 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82872abc-15ff-4e2e-aff7-fbc0a956a29f None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:29.181602 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9a6d9-4020-4fa8-89c7-c58fbf689870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:29.182525 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9a6d9-4020-4fa8-89c7-c58fbf689870 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:29.183013 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9a6d9-4020-4fa8-89c7-c58fbf689870 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:29.183436 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9a6d9-4020-4fa8-89c7-c58fbf689870 None admin] PATH_INFO: `/v3/projects/54786471df9a4027b37cc1f2e82d05e6` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:29.184829 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f9a6d9-4020-4fa8-89c7-c58fbf689870 None admin] RBAC: Authorizing `identity:get_project(project_id=54786471df9a4027b37cc1f2e82d05e6)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:29.187166 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f9a6d9-4020-4fa8-89c7-c58fbf689870 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:29.189874 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 44/95] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:47:29 2022] GET /identity/v3/projects/54786471df9a4027b37cc1f2e82d05e6 => generated 312 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:47:29.201593 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9a6d9-4020-4fa8-89c7-c58fbf689870 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:29.231593 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a6b441-3d51-4ed4-88ef-c9bbb209993c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:29.233117 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6b441-3d51-4ed4-88ef-c9bbb209993c None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:29.234537 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6b441-3d51-4ed4-88ef-c9bbb209993c None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:29.235132 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6b441-3d51-4ed4-88ef-c9bbb209993c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:29.236188 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a6b441-3d51-4ed4-88ef-c9bbb209993c None admin] RBAC: Adding query filter params (role.id=7df9ebc56a2a4214b2b64a5941812ef0, scope.project.id=c8fe5139a2a24ec780814c7f39d5b437, user.id=43855050fbcd47d3b5ded0e8ac79e20d) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:29.237708 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a6b441-3d51-4ed4-88ef-c9bbb209993c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:29.240263 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a6b441-3d51-4ed4-88ef-c9bbb209993c None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:29.252098 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 52/96] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:47:28 2022] GET /identity/v3/role_assignments?user.id=43855050fbcd47d3b5ded0e8ac79e20d&scope.project.id=c8fe5139a2a24ec780814c7f39d5b437&role.id=7df9ebc56a2a4214b2b64a5941812ef0 => generated 259 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:29.295957 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:29.373857 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9a6d9-4020-4fa8-89c7-c58fbf689870 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:29.420894 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d76450b-417a-4ee6-8e11-95e1b4d61daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:29.421906 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76450b-417a-4ee6-8e11-95e1b4d61daf None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:29.422392 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76450b-417a-4ee6-8e11-95e1b4d61daf None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:29.422865 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76450b-417a-4ee6-8e11-95e1b4d61daf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:29.423900 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d76450b-417a-4ee6-8e11-95e1b4d61daf None admin] RBAC: Adding query filter params (role.id=d1261c99e1d64f9da63adfff739561e8, scope.project.id=54786471df9a4027b37cc1f2e82d05e6, user.id=43855050fbcd47d3b5ded0e8ac79e20d) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:29.424426 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d76450b-417a-4ee6-8e11-95e1b4d61daf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:29.426097 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d76450b-417a-4ee6-8e11-95e1b4d61daf None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:29.436896 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 45/97] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:47:29 2022] GET /identity/v3/role_assignments?user.id=43855050fbcd47d3b5ded0e8ac79e20d&scope.project.id=54786471df9a4027b37cc1f2e82d05e6&role.id=d1261c99e1d64f9da63adfff739561e8 => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:30.733438 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881f9a2-ba71-403d-9851-a652e2b999bf None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:30.734447 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881f9a2-ba71-403d-9851-a652e2b999bf None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:30.734948 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881f9a2-ba71-403d-9851-a652e2b999bf None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:30.736765 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 53/98] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:30.742997 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db730e60-8741-425d-a6a3-a396dde6c9be None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:30.743880 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db730e60-8741-425d-a6a3-a396dde6c9be None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:30.744361 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db730e60-8741-425d-a6a3-a396dde6c9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:30.763559 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-db730e60-8741-425d-a6a3-a396dde6c9be None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:30.765114 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-db730e60-8741-425d-a6a3-a396dde6c9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:30.862978 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 46/99] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:30.966561 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbe497-54b8-453b-b0a2-12ada15fb2f1 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:30.967189 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbe497-54b8-453b-b0a2-12ada15fb2f1 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:30.967675 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbe497-54b8-453b-b0a2-12ada15fb2f1 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:30.969266 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 54/100] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:30.978343 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4834492-1d33-4d3a-8ef3-b1722d96e6eb None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:30.978638 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4834492-1d33-4d3a-8ef3-b1722d96e6eb None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:30.978912 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4834492-1d33-4d3a-8ef3-b1722d96e6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:31.003475 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f4834492-1d33-4d3a-8ef3-b1722d96e6eb None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:31.004697 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f4834492-1d33-4d3a-8ef3-b1722d96e6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:31.045873 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:31.091792 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 47/101] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:31.141309 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dba445-851c-4774-8e17-4ca15f35a7d3 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:31.141926 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dba445-851c-4774-8e17-4ca15f35a7d3 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:31.142408 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dba445-851c-4774-8e17-4ca15f35a7d3 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:31.144201 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 55/102] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:31.150380 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3febf6dd-4b0d-4fd5-8f3a-264a0086bad5 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:31.150924 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3febf6dd-4b0d-4fd5-8f3a-264a0086bad5 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:31.151416 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3febf6dd-4b0d-4fd5-8f3a-264a0086bad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:31.170574 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-3febf6dd-4b0d-4fd5-8f3a-264a0086bad5 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:31.172070 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3febf6dd-4b0d-4fd5-8f3a-264a0086bad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:31.231862 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dba445-851c-4774-8e17-4ca15f35a7d3 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:31.285052 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 48/103] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:31.359383 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dba445-851c-4774-8e17-4ca15f35a7d3 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:31.380684 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:31.403062 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d93e5-36dc-42cb-99c0-938145481183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:31.403822 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d93e5-36dc-42cb-99c0-938145481183 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:31.404113 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d93e5-36dc-42cb-99c0-938145481183 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:31.404392 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d93e5-36dc-42cb-99c0-938145481183 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:31.417763 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805d93e5-36dc-42cb-99c0-938145481183 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:47:31.422758 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-805d93e5-36dc-42cb-99c0-938145481183 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 00:47:31.422758 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:31.422758 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:31.422758 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:31.422758 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:31.422758 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:31.422758 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:31.422758 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:31.422758 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:31.422758 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:31.422758 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:31.422758 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:31.422758 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:47:31.422758 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:47:31.422758 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:47:31.422758 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:47:31.423675 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:47:31.423675 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:47:31.423675 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:47:31.423675 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:47:31.423675 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:31.423675 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:31.423675 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:31.423675 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:31.423675 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:31.423675 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:31.423675 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:31.423675 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:31.423675 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:31.423675 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:31.423675 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:31.423675 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:31.424398 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:31.424398 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:31.424398 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:31.424398 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:31.424398 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:47:31.424398 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:47:31.424398 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:47:31.424398 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:47:31.424398 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:47:31.424398 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:47:31.424398 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 00:47:31.424398 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:47:31.424398 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 56/104] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:47:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:47:31.430046 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3febf6dd-4b0d-4fd5-8f3a-264a0086bad5 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:31.485041 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d93e5-36dc-42cb-99c0-938145481183 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:31.542283 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:31.545800 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3febf6dd-4b0d-4fd5-8f3a-264a0086bad5 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:31.609897 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877446f9-103b-4de1-be97-220e2730451c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:31.610755 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877446f9-103b-4de1-be97-220e2730451c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:31.611175 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877446f9-103b-4de1-be97-220e2730451c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:31.611575 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877446f9-103b-4de1-be97-220e2730451c None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:31.612382 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877446f9-103b-4de1-be97-220e2730451c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:31.612866 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877446f9-103b-4de1-be97-220e2730451c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:31.614381 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877446f9-103b-4de1-be97-220e2730451c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:31.631897 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 49/105] 10.222.0.27 () {56 vars in 1101 bytes} [Sun Sep 4 00:47:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:31.637754 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877446f9-103b-4de1-be97-220e2730451c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:31.664809 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d93e5-36dc-42cb-99c0-938145481183 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:31.731988 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0db65-51bb-43fe-9db9-cecdfb84e484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:31.732876 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0db65-51bb-43fe-9db9-cecdfb84e484 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:31.733294 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0db65-51bb-43fe-9db9-cecdfb84e484 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:31.733703 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0db65-51bb-43fe-9db9-cecdfb84e484 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:31.738000 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af0db65-51bb-43fe-9db9-cecdfb84e484 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:31.739621 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af0db65-51bb-43fe-9db9-cecdfb84e484 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:31.743416 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 57/106] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:47:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:47:31.751360 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0db65-51bb-43fe-9db9-cecdfb84e484 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:31.836176 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877446f9-103b-4de1-be97-220e2730451c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:31.848815 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:31.901535 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a7b6c-bbe9-4da7-9ecd-7a60a21447fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:31.902240 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a7b6c-bbe9-4da7-9ecd-7a60a21447fa None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:31.902524 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a7b6c-bbe9-4da7-9ecd-7a60a21447fa None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:31.902810 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a7b6c-bbe9-4da7-9ecd-7a60a21447fa None admin] PATH_INFO: `/v3/users/5511ff4386ee4bb38c97add3954b95e0` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:31.913136 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0db65-51bb-43fe-9db9-cecdfb84e484 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:31.944267 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a7b6c-bbe9-4da7-9ecd-7a60a21447fa None admin] RBAC: Authorizing `identity:get_user(user_id=5511ff4386ee4bb38c97add3954b95e0)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:31.945790 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a7b6c-bbe9-4da7-9ecd-7a60a21447fa None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:31.948997 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 50/107] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:47:31 2022] GET /identity/v3/users/5511ff4386ee4bb38c97add3954b95e0 => generated 289 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:47:31.950172 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a7b6c-bbe9-4da7-9ecd-7a60a21447fa None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:31.952581 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1beb09-b032-43f1-b5cf-111cb5deb50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:31.952581 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1beb09-b032-43f1-b5cf-111cb5deb50b None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:31.952581 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1beb09-b032-43f1-b5cf-111cb5deb50b None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:31.952581 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1beb09-b032-43f1-b5cf-111cb5deb50b None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:31.952898 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1beb09-b032-43f1-b5cf-111cb5deb50b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:31.956492 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1beb09-b032-43f1-b5cf-111cb5deb50b None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:31.985259 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:32.035196 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 58/108] 10.222.0.27 () {60 vars in 1141 bytes} [Sun Sep 4 00:47:31 2022] POST /identity/v3/projects => generated 302 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:47:32.037343 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1beb09-b032-43f1-b5cf-111cb5deb50b None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:32.110531 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a7b6c-bbe9-4da7-9ecd-7a60a21447fa None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:32.174755 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb4c23d-f3cd-46b7-9204-f8e1df386ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:32.177286 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4c23d-f3cd-46b7-9204-f8e1df386ac0 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:32.178049 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4c23d-f3cd-46b7-9204-f8e1df386ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:32.178675 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4c23d-f3cd-46b7-9204-f8e1df386ac0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:32.209247 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb4c23d-f3cd-46b7-9204-f8e1df386ac0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:47:32.214811 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-aeb4c23d-f3cd-46b7-9204-f8e1df386ac0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:47:32.214811 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:32.214811 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:32.214811 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:32.214811 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:32.214811 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:32.214811 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:32.214811 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:32.214811 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:32.214811 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:32.214811 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:32.214811 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:32.214811 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:47:32.214811 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:47:32.214811 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:47:32.214811 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:47:32.215654 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:47:32.215654 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:47:32.215654 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:47:32.215654 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:47:32.215654 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:32.215654 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:32.215654 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:32.215654 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:32.215654 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:32.215654 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:32.215654 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:32.215654 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:32.215654 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:32.215654 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:32.215654 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:32.215654 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:32.216393 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:32.216393 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:32.216393 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:32.216393 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:32.216393 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:47:32.216393 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:47:32.216393 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:47:32.216393 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:47:32.216393 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:47:32.216393 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:47:32.216393 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:47:32.216393 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:47:32.216393 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 51/109] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:47:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:47:32.217774 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ecfaa-3161-4648-b4c2-1cd9d7bba12c None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:32.219325 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ecfaa-3161-4648-b4c2-1cd9d7bba12c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:32.222520 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ecfaa-3161-4648-b4c2-1cd9d7bba12c None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:32.222520 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 52/110] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:32.233404 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4aa29-fe3d-4050-a221-96831bf2c0d5 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:32.233404 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4aa29-fe3d-4050-a221-96831bf2c0d5 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:32.233404 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4aa29-fe3d-4050-a221-96831bf2c0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:32.256432 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:32.260602 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1beb09-b032-43f1-b5cf-111cb5deb50b None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:32.265831 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-b3b4aa29-fe3d-4050-a221-96831bf2c0d5 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:32.267363 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b3b4aa29-fe3d-4050-a221-96831bf2c0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:32.299662 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ea79b-1446-45ae-a8ed-bc48fbdd85ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:32.300716 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ea79b-1446-45ae-a8ed-bc48fbdd85ce None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:32.301075 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ea79b-1446-45ae-a8ed-bc48fbdd85ce None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:32.301464 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ea79b-1446-45ae-a8ed-bc48fbdd85ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:32.344187 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 53/111] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:32.345327 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4aa29-fe3d-4050-a221-96831bf2c0d5 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:32.360226 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:32.381504 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ea79b-1446-45ae-a8ed-bc48fbdd85ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:32.383405 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ea79b-1446-45ae-a8ed-bc48fbdd85ce None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:32.470270 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4aa29-fe3d-4050-a221-96831bf2c0d5 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:32.499314 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:32.505217 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 59/112] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:47:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:47:32.507738 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb10099-2618-407c-9259-a84ab2f250de None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:32.508214 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb10099-2618-407c-9259-a84ab2f250de None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:32.511542 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb10099-2618-407c-9259-a84ab2f250de None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:32.511542 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 60/113] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:32.512420 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb10099-2618-407c-9259-a84ab2f250de None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:32.529238 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648499a3-3271-4ae8-ae9b-129fcf59b05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:32.529870 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648499a3-3271-4ae8-ae9b-129fcf59b05f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:32.530302 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648499a3-3271-4ae8-ae9b-129fcf59b05f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:32.530587 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648499a3-3271-4ae8-ae9b-129fcf59b05f None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:32.531654 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648499a3-3271-4ae8-ae9b-129fcf59b05f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:32.531984 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648499a3-3271-4ae8-ae9b-129fcf59b05f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:32.533668 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648499a3-3271-4ae8-ae9b-129fcf59b05f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:32.554634 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 54/114] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:47:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:32.557632 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffba9d8-344d-455b-9f04-c35030e71979 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:32.559310 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffba9d8-344d-455b-9f04-c35030e71979 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:32.560489 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffba9d8-344d-455b-9f04-c35030e71979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:32.595684 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-2ffba9d8-344d-455b-9f04-c35030e71979 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:32.598955 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2ffba9d8-344d-455b-9f04-c35030e71979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:32.684623 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 55/115] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:32.685701 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffba9d8-344d-455b-9f04-c35030e71979 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:32.688275 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:32.716489 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb10099-2618-407c-9259-a84ab2f250de None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:32.761488 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd810a8-bd89-43fb-8a8d-9f7cb79630c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:32.762261 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd810a8-bd89-43fb-8a8d-9f7cb79630c5 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:32.762472 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd810a8-bd89-43fb-8a8d-9f7cb79630c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:32.762808 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd810a8-bd89-43fb-8a8d-9f7cb79630c5 None admin] PATH_INFO: `/v3/projects/c8fe5139a2a24ec780814c7f39d5b437` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:32.764056 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd810a8-bd89-43fb-8a8d-9f7cb79630c5 None admin] RBAC: Authorizing `identity:get_project(project_id=c8fe5139a2a24ec780814c7f39d5b437)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:32.768708 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd810a8-bd89-43fb-8a8d-9f7cb79630c5 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:32.770628 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 61/116] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:47:32 2022] GET /identity/v3/projects/c8fe5139a2a24ec780814c7f39d5b437 => generated 298 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:32.774440 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c6b396-1515-41c5-8e95-ea5c9d9d1f9e None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:32.776874 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c6b396-1515-41c5-8e95-ea5c9d9d1f9e None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:32.776874 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c6b396-1515-41c5-8e95-ea5c9d9d1f9e None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:32.776874 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 62/117] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:32.777643 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c6b396-1515-41c5-8e95-ea5c9d9d1f9e None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:32.799959 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffba9d8-344d-455b-9f04-c35030e71979 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:32.839862 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba172af-8d4e-4bba-841c-4a67b116441a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:32.840651 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba172af-8d4e-4bba-841c-4a67b116441a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:32.840899 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba172af-8d4e-4bba-841c-4a67b116441a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:32.841076 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba172af-8d4e-4bba-841c-4a67b116441a None admin] PATH_INFO: `/v3/users/644f0b1b120742d2aca15aeeaa76abcc` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:32.845217 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba172af-8d4e-4bba-841c-4a67b116441a None admin] RBAC: Authorizing `identity:get_user(user_id=644f0b1b120742d2aca15aeeaa76abcc)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:32.847098 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba172af-8d4e-4bba-841c-4a67b116441a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:32.849179 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 56/118] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:47:32 2022] GET /identity/v3/users/644f0b1b120742d2aca15aeeaa76abcc => generated 247 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:32.851040 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba11d2-b502-4748-a5ea-2afcd0d24d1c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:32.851588 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba11d2-b502-4748-a5ea-2afcd0d24d1c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:32.852176 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba11d2-b502-4748-a5ea-2afcd0d24d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:32.873047 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-71ba11d2-b502-4748-a5ea-2afcd0d24d1c None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:32.875042 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:32.875526 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-71ba11d2-b502-4748-a5ea-2afcd0d24d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:32.920186 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c6b396-1515-41c5-8e95-ea5c9d9d1f9e None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:32.951428 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:32.956642 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 57/119] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:32.957775 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba11d2-b502-4748-a5ea-2afcd0d24d1c None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:32.965828 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106718b-c49b-45c4-b2c0-3290272c42e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:32.966717 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106718b-c49b-45c4-b2c0-3290272c42e7 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:32.967233 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106718b-c49b-45c4-b2c0-3290272c42e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:32.967689 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106718b-c49b-45c4-b2c0-3290272c42e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:32.968610 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6106718b-c49b-45c4-b2c0-3290272c42e7 None admin] RBAC: Adding query filter params (role.id=8f7a2ac695c6482cbf48819cee250c22, scope.project.id=c8fe5139a2a24ec780814c7f39d5b437, user.id=5511ff4386ee4bb38c97add3954b95e0) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:32.969089 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6106718b-c49b-45c4-b2c0-3290272c42e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:32.970725 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6106718b-c49b-45c4-b2c0-3290272c42e7 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:32.977094 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 63/120] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:47:32 2022] GET /identity/v3/role_assignments?user.id=5511ff4386ee4bb38c97add3954b95e0&scope.project.id=c8fe5139a2a24ec780814c7f39d5b437&role.id=8f7a2ac695c6482cbf48819cee250c22 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:32.978401 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106718b-c49b-45c4-b2c0-3290272c42e7 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:33.141309 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba11d2-b502-4748-a5ea-2afcd0d24d1c None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:33.155438 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106718b-c49b-45c4-b2c0-3290272c42e7 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:33.191087 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:33.198063 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4871d30-2480-455e-a9a7-244276b5d6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:33.199270 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4871d30-2480-455e-a9a7-244276b5d6a5 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:33.199749 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4871d30-2480-455e-a9a7-244276b5d6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:33.200217 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4871d30-2480-455e-a9a7-244276b5d6a5 None admin] PATH_INFO: `/v3/projects/c8fe5139a2a24ec780814c7f39d5b437` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:33.201723 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4871d30-2480-455e-a9a7-244276b5d6a5 None admin] RBAC: Authorizing `identity:get_project(project_id=c8fe5139a2a24ec780814c7f39d5b437)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:33.203380 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4871d30-2480-455e-a9a7-244276b5d6a5 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:33.204211 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d37cde4-262e-4299-a8e2-baa02c08f684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:33.205074 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37cde4-262e-4299-a8e2-baa02c08f684 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:33.208519 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37cde4-262e-4299-a8e2-baa02c08f684 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:33.209167 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 64/121] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:47:32 2022] GET /identity/v3/projects/c8fe5139a2a24ec780814c7f39d5b437 => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:33.209270 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37cde4-262e-4299-a8e2-baa02c08f684 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:33.209417 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4871d30-2480-455e-a9a7-244276b5d6a5 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:33.218687 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d37cde4-262e-4299-a8e2-baa02c08f684 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:47:33.223874 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-0d37cde4-262e-4299-a8e2-baa02c08f684 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:47:33.223874 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:33.223874 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:33.223874 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:33.223874 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:33.223874 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:33.223874 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:33.223874 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:33.223874 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:33.223874 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:33.223874 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:33.223874 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:33.223874 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:47:33.223874 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:47:33.223874 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:47:33.223874 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:47:33.225302 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:47:33.225302 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:47:33.225302 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:47:33.225302 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:47:33.225302 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:33.225302 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:33.225302 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:33.225302 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:33.225302 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:33.225302 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:33.225302 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:33.225302 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:33.225302 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:33.225302 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:33.225302 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:33.225302 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:33.226303 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:33.226303 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:33.226303 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:33.226303 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:33.226303 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:47:33.226303 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:47:33.226303 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:47:33.226303 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:47:33.226303 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:47:33.226303 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:47:33.226303 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:47:33.226303 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:47:33.232577 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 58/122] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:47:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:47:33.234462 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d37cde4-262e-4299-a8e2-baa02c08f684 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:33.312830 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:33.370469 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d37cde4-262e-4299-a8e2-baa02c08f684 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:33.373208 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4871d30-2480-455e-a9a7-244276b5d6a5 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:33.412623 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76223b2a-4c8c-4d9f-bd92-ee17500224a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:33.413705 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76223b2a-4c8c-4d9f-bd92-ee17500224a8 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:33.414136 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76223b2a-4c8c-4d9f-bd92-ee17500224a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:33.414554 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76223b2a-4c8c-4d9f-bd92-ee17500224a8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:33.419580 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c99be-0433-47a4-b69c-85493f54a557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:33.420386 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c99be-0433-47a4-b69c-85493f54a557 None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:33.420795 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c99be-0433-47a4-b69c-85493f54a557 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:33.421173 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c99be-0433-47a4-b69c-85493f54a557 None admin] PATH_INFO: `/v3/projects/c8fe5139a2a24ec780814c7f39d5b437/users/644f0b1b120742d2aca15aeeaa76abcc/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:33.423529 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0c99be-0433-47a4-b69c-85493f54a557 None admin] RBAC: Authorizing `identity:create_grant(project_id=c8fe5139a2a24ec780814c7f39d5b437, user_id=644f0b1b120742d2aca15aeeaa76abcc, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:33.425879 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0c99be-0433-47a4-b69c-85493f54a557 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:33.429461 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76223b2a-4c8c-4d9f-bd92-ee17500224a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:47:33.435699 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-76223b2a-4c8c-4d9f-bd92-ee17500224a8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 00:47:33.435699 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:33.435699 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:33.435699 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:33.435699 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:33.435699 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:33.435699 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:33.435699 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:33.435699 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:33.435699 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:33.435699 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:33.435699 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:33.435699 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:47:33.435699 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:47:33.435699 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:47:33.435699 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:47:33.436661 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:47:33.436661 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:47:33.436661 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:47:33.436661 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:47:33.436661 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:33.436661 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:33.436661 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:33.436661 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:33.436661 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:33.436661 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:33.436661 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:33.436661 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:33.436661 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:33.436661 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:33.436661 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:33.436661 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:33.437419 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:33.437419 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:33.437419 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:33.437419 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:33.437419 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:47:33.437419 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:47:33.437419 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:47:33.437419 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:47:33.437419 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:47:33.437419 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:47:33.437419 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 00:47:33.437419 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:47:33.442922 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 65/123] 10.222.0.27 () {56 vars in 1114 bytes} [Sun Sep 4 00:47:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:47:33.447283 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76223b2a-4c8c-4d9f-bd92-ee17500224a8 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:33.502384 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 59/124] 10.222.0.27 () {58 vars in 1439 bytes} [Sun Sep 4 00:47:33 2022] PUT /identity/v3/projects/c8fe5139a2a24ec780814c7f39d5b437/users/644f0b1b120742d2aca15aeeaa76abcc/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 00:47:33.503243 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c99be-0433-47a4-b69c-85493f54a557 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:33.520450 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:33.586603 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76223b2a-4c8c-4d9f-bd92-ee17500224a8 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:33.624980 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c99be-0433-47a4-b69c-85493f54a557 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:33.631609 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d24dda-f73c-48eb-b1d4-1f8b913520c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:33.632417 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d24dda-f73c-48eb-b1d4-1f8b913520c6 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:33.632832 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d24dda-f73c-48eb-b1d4-1f8b913520c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:33.633256 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d24dda-f73c-48eb-b1d4-1f8b913520c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:33.634043 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d24dda-f73c-48eb-b1d4-1f8b913520c6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:33.634481 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d24dda-f73c-48eb-b1d4-1f8b913520c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:33.636002 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d24dda-f73c-48eb-b1d4-1f8b913520c6 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:33.646751 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 66/125] 10.222.0.27 () {56 vars in 1101 bytes} [Sun Sep 4 00:47:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:33.652412 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d24dda-f73c-48eb-b1d4-1f8b913520c6 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:33.654269 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:33.675455 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377955d7-9405-4e9d-a7ad-e73986035253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:33.676335 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377955d7-9405-4e9d-a7ad-e73986035253 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:33.676781 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377955d7-9405-4e9d-a7ad-e73986035253 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:33.677239 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377955d7-9405-4e9d-a7ad-e73986035253 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:33.696171 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377955d7-9405-4e9d-a7ad-e73986035253 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:47:33.702331 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-377955d7-9405-4e9d-a7ad-e73986035253 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:47:33.702331 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:33.702331 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:33.702331 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:33.702331 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:33.702331 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:33.702331 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:33.702331 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:33.702331 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:33.702331 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:33.702331 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:33.702331 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:33.702331 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:47:33.702331 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:47:33.702331 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:47:33.702331 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:47:33.703391 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:47:33.703391 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:47:33.703391 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:47:33.703391 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:47:33.703391 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:33.703391 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:33.703391 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:33.703391 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:33.703391 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:33.703391 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:33.703391 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:33.703391 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:33.703391 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:33.703391 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:33.703391 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:33.703391 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:33.704341 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:33.704341 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:33.704341 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:33.704341 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:33.704341 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:47:33.704341 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:47:33.704341 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:47:33.704341 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:47:33.704341 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:47:33.704341 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:47:33.704341 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:47:33.704341 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:47:33.707092 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 60/126] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:47:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:47:33.708172 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377955d7-9405-4e9d-a7ad-e73986035253 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:33.789345 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d24dda-f73c-48eb-b1d4-1f8b913520c6 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:33.814968 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:33.830629 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff5043-cef0-4c58-bcf6-7078e66a9062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:33.831348 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff5043-cef0-4c58-bcf6-7078e66a9062 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:33.831641 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff5043-cef0-4c58-bcf6-7078e66a9062 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:33.831926 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff5043-cef0-4c58-bcf6-7078e66a9062 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:33.832955 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff5043-cef0-4c58-bcf6-7078e66a9062 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:33.833273 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff5043-cef0-4c58-bcf6-7078e66a9062 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:33.835170 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff5043-cef0-4c58-bcf6-7078e66a9062 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:33.844149 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 67/127] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:47:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:33.844815 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff5043-cef0-4c58-bcf6-7078e66a9062 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:33.847845 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377955d7-9405-4e9d-a7ad-e73986035253 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:33.891101 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f60e9-bace-447f-81dc-27be864c2a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:33.892131 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f60e9-bace-447f-81dc-27be864c2a6b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:33.892383 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f60e9-bace-447f-81dc-27be864c2a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:33.892812 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f60e9-bace-447f-81dc-27be864c2a6b None admin] PATH_INFO: `/v3/users/43855050fbcd47d3b5ded0e8ac79e20d` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:33.894043 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234f60e9-bace-447f-81dc-27be864c2a6b None admin] RBAC: Authorizing `identity:get_user(user_id=43855050fbcd47d3b5ded0e8ac79e20d)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:33.895644 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234f60e9-bace-447f-81dc-27be864c2a6b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:33.897592 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 61/128] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:47:33 2022] GET /identity/v3/users/43855050fbcd47d3b5ded0e8ac79e20d => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:33.898687 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f60e9-bace-447f-81dc-27be864c2a6b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:33.957173 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:33.979894 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff5043-cef0-4c58-bcf6-7078e66a9062 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:34.015686 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f60e9-bace-447f-81dc-27be864c2a6b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:34.025154 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db799581-38dc-46c7-b32b-ddfa371cef4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:34.026342 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db799581-38dc-46c7-b32b-ddfa371cef4c None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:34.026773 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db799581-38dc-46c7-b32b-ddfa371cef4c None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:34.027187 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db799581-38dc-46c7-b32b-ddfa371cef4c None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:34.029788 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db799581-38dc-46c7-b32b-ddfa371cef4c None admin] RBAC: Adding query filter params (name=member) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:34.030145 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db799581-38dc-46c7-b32b-ddfa371cef4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:34.031580 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db799581-38dc-46c7-b32b-ddfa371cef4c None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:34.041757 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 68/129] 10.222.0.27 () {56 vars in 1101 bytes} [Sun Sep 4 00:47:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:34.042335 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db799581-38dc-46c7-b32b-ddfa371cef4c None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:34.061861 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb4503-15ba-4709-80fc-707a8b02f4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:34.063044 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb4503-15ba-4709-80fc-707a8b02f4bd None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:34.063813 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb4503-15ba-4709-80fc-707a8b02f4bd None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:34.064292 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb4503-15ba-4709-80fc-707a8b02f4bd None admin] PATH_INFO: `/v3/users/43855050fbcd47d3b5ded0e8ac79e20d` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:34.065644 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedb4503-15ba-4709-80fc-707a8b02f4bd None admin] RBAC: Authorizing `identity:get_user(user_id=43855050fbcd47d3b5ded0e8ac79e20d)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:34.067920 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedb4503-15ba-4709-80fc-707a8b02f4bd None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:34.070272 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 62/130] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:47:33 2022] GET /identity/v3/users/43855050fbcd47d3b5ded0e8ac79e20d => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:34.071621 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb4503-15ba-4709-80fc-707a8b02f4bd None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:34.098441 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:34.178263 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db799581-38dc-46c7-b32b-ddfa371cef4c None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:34.196346 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb4503-15ba-4709-80fc-707a8b02f4bd None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:34.216805 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:34.236664 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf2bd2-2217-4919-bb85-ffaf494f9aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:34.237668 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf2bd2-2217-4919-bb85-ffaf494f9aea None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:34.238172 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf2bd2-2217-4919-bb85-ffaf494f9aea None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:34.238773 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf2bd2-2217-4919-bb85-ffaf494f9aea None admin] PATH_INFO: `/v3/users/43855050fbcd47d3b5ded0e8ac79e20d` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:34.240784 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcf2bd2-2217-4919-bb85-ffaf494f9aea None admin] RBAC: Authorizing `identity:get_user(user_id=43855050fbcd47d3b5ded0e8ac79e20d)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:34.242846 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcf2bd2-2217-4919-bb85-ffaf494f9aea None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:34.244981 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 63/131] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:47:34 2022] GET /identity/v3/users/43855050fbcd47d3b5ded0e8ac79e20d => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:34.246111 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf2bd2-2217-4919-bb85-ffaf494f9aea None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:34.270695 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18754b0-dbf4-4473-84cd-4f862c8f650f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:34.272081 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18754b0-dbf4-4473-84cd-4f862c8f650f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:34.272646 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18754b0-dbf4-4473-84cd-4f862c8f650f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:34.272937 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18754b0-dbf4-4473-84cd-4f862c8f650f None admin] PATH_INFO: `/v3/projects/c8fe5139a2a24ec780814c7f39d5b437` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:34.279822 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18754b0-dbf4-4473-84cd-4f862c8f650f None admin] RBAC: Authorizing `identity:get_project(project_id=c8fe5139a2a24ec780814c7f39d5b437)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:34.281406 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18754b0-dbf4-4473-84cd-4f862c8f650f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:34.283748 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 69/132] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:47:34 2022] GET /identity/v3/projects/c8fe5139a2a24ec780814c7f39d5b437 => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:34.284647 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18754b0-dbf4-4473-84cd-4f862c8f650f None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:34.415072 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf2bd2-2217-4919-bb85-ffaf494f9aea None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:34.437480 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18754b0-dbf4-4473-84cd-4f862c8f650f None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:34.464485 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:34.470670 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d82299-fc16-461b-bd5d-529668a3dcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:34.471312 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d82299-fc16-461b-bd5d-529668a3dcd8 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:34.471597 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d82299-fc16-461b-bd5d-529668a3dcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:34.471875 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d82299-fc16-461b-bd5d-529668a3dcd8 None admin] PATH_INFO: `/v3/projects/c8fe5139a2a24ec780814c7f39d5b437` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:34.476273 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d82299-fc16-461b-bd5d-529668a3dcd8 None admin] RBAC: Authorizing `identity:get_project(project_id=c8fe5139a2a24ec780814c7f39d5b437)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:34.477762 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d82299-fc16-461b-bd5d-529668a3dcd8 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:34.481012 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 64/133] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:47:34 2022] GET /identity/v3/projects/c8fe5139a2a24ec780814c7f39d5b437 => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:34.481761 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d82299-fc16-461b-bd5d-529668a3dcd8 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:34.505665 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c47f2-b4b2-4108-9716-0105de26d15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:34.506306 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c47f2-b4b2-4108-9716-0105de26d15c None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:34.506604 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c47f2-b4b2-4108-9716-0105de26d15c None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:34.506875 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c47f2-b4b2-4108-9716-0105de26d15c None admin] PATH_INFO: `/v3/projects/54786471df9a4027b37cc1f2e82d05e6` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:34.508186 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90c47f2-b4b2-4108-9716-0105de26d15c None admin] RBAC: Authorizing `identity:get_project(project_id=54786471df9a4027b37cc1f2e82d05e6)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:34.509677 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90c47f2-b4b2-4108-9716-0105de26d15c None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:34.511637 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 70/134] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:47:34 2022] GET /identity/v3/projects/54786471df9a4027b37cc1f2e82d05e6 => generated 312 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:34.512399 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c47f2-b4b2-4108-9716-0105de26d15c None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:34.588493 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:34.673017 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d82299-fc16-461b-bd5d-529668a3dcd8 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:34.684795 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c47f2-b4b2-4108-9716-0105de26d15c None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:34.721397 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1870b0be-bcc5-4712-b815-edad0bb1b0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:34.722067 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870b0be-bcc5-4712-b815-edad0bb1b0fd None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:34.722354 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870b0be-bcc5-4712-b815-edad0bb1b0fd None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:34.722454 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1439a353-1b8e-4542-a9cd-35257443db93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:34.722624 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870b0be-bcc5-4712-b815-edad0bb1b0fd None admin] PATH_INFO: `/v3/projects/c8fe5139a2a24ec780814c7f39d5b437/users/43855050fbcd47d3b5ded0e8ac79e20d/roles/7df9ebc56a2a4214b2b64a5941812ef0` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:34.723173 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439a353-1b8e-4542-a9cd-35257443db93 None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:34.723468 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439a353-1b8e-4542-a9cd-35257443db93 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:34.723750 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439a353-1b8e-4542-a9cd-35257443db93 None admin] PATH_INFO: `/v3/projects/c8fe5139a2a24ec780814c7f39d5b437/users/43855050fbcd47d3b5ded0e8ac79e20d/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:34.725339 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1870b0be-bcc5-4712-b815-edad0bb1b0fd None admin] RBAC: Authorizing `identity:create_grant(project_id=c8fe5139a2a24ec780814c7f39d5b437, user_id=43855050fbcd47d3b5ded0e8ac79e20d, role_id=7df9ebc56a2a4214b2b64a5941812ef0)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:34.726382 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1439a353-1b8e-4542-a9cd-35257443db93 None admin] RBAC: Authorizing `identity:create_grant(project_id=c8fe5139a2a24ec780814c7f39d5b437, user_id=43855050fbcd47d3b5ded0e8ac79e20d, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:34.726796 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1870b0be-bcc5-4712-b815-edad0bb1b0fd None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:34.727831 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1439a353-1b8e-4542-a9cd-35257443db93 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:34.830287 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 65/135] 10.222.0.27 () {58 vars in 1439 bytes} [Sun Sep 4 00:47:34 2022] PUT /identity/v3/projects/c8fe5139a2a24ec780814c7f39d5b437/users/43855050fbcd47d3b5ded0e8ac79e20d/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 00:47:34.831112 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1439a353-1b8e-4542-a9cd-35257443db93 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:34.864075 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 71/136] 10.222.0.27 () {58 vars in 1439 bytes} [Sun Sep 4 00:47:34 2022] PUT /identity/v3/projects/c8fe5139a2a24ec780814c7f39d5b437/users/43855050fbcd47d3b5ded0e8ac79e20d/roles/7df9ebc56a2a4214b2b64a5941812ef0 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 00:47:34.866610 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdb2f5-7019-4f3c-b5cd-7d4397d63815 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:34.866912 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdb2f5-7019-4f3c-b5cd-7d4397d63815 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:34.867195 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdb2f5-7019-4f3c-b5cd-7d4397d63815 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:34.869498 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 72/137] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:34.875950 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2362c-1c4a-484d-9876-442e34238ba1 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:34.876270 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2362c-1c4a-484d-9876-442e34238ba1 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:34.876543 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2362c-1c4a-484d-9876-442e34238ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:34.916840 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-88d2362c-1c4a-484d-9876-442e34238ba1 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:34.920352 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-88d2362c-1c4a-484d-9876-442e34238ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:35.004043 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:35.045904 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1439a353-1b8e-4542-a9cd-35257443db93 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:35.055944 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 73/138] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 00:47:35.067577 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:35.093814 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85255c57-ab0e-47e9-bbdb-b0166e102ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:35.094491 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85255c57-ab0e-47e9-bbdb-b0166e102ab2 None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:35.094773 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85255c57-ab0e-47e9-bbdb-b0166e102ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:35.095047 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85255c57-ab0e-47e9-bbdb-b0166e102ab2 None admin] PATH_INFO: `/v3/projects/54786471df9a4027b37cc1f2e82d05e6/users/43855050fbcd47d3b5ded0e8ac79e20d/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:35.096990 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85255c57-ab0e-47e9-bbdb-b0166e102ab2 None admin] RBAC: Authorizing `identity:create_grant(project_id=54786471df9a4027b37cc1f2e82d05e6, user_id=43855050fbcd47d3b5ded0e8ac79e20d, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:35.098504 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85255c57-ab0e-47e9-bbdb-b0166e102ab2 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:35.174539 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 66/139] 10.222.0.27 () {58 vars in 1439 bytes} [Sun Sep 4 00:47:34 2022] PUT /identity/v3/projects/54786471df9a4027b37cc1f2e82d05e6/users/43855050fbcd47d3b5ded0e8ac79e20d/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 00:47:35.352793 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a5043-bfbd-437f-b5cb-116680a23b1c None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:35.353144 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a5043-bfbd-437f-b5cb-116680a23b1c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:35.353333 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a5043-bfbd-437f-b5cb-116680a23b1c None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:35.354902 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 74/140] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:35.365622 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72799b-fb86-429a-931f-cc67e68cf11a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:35.365949 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72799b-fb86-429a-931f-cc67e68cf11a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:35.366171 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72799b-fb86-429a-931f-cc67e68cf11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:35.390306 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-eb72799b-fb86-429a-931f-cc67e68cf11a None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:35.391516 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-eb72799b-fb86-429a-931f-cc67e68cf11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:35.426126 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a5043-bfbd-437f-b5cb-116680a23b1c None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:35.516199 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 67/141] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 00:47:35.568081 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a5043-bfbd-437f-b5cb-116680a23b1c None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:35.604948 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f65863-3e39-4ca6-a6f2-e61fd4e2bd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:35.605509 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f65863-3e39-4ca6-a6f2-e61fd4e2bd4c None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:35.605790 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f65863-3e39-4ca6-a6f2-e61fd4e2bd4c None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:35.606067 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f65863-3e39-4ca6-a6f2-e61fd4e2bd4c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:35.607175 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f65863-3e39-4ca6-a6f2-e61fd4e2bd4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:35.608673 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f65863-3e39-4ca6-a6f2-e61fd4e2bd4c None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:35.610240 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 75/142] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:47:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:47:35.616910 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72799b-fb86-429a-931f-cc67e68cf11a None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:35.789094 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72799b-fb86-429a-931f-cc67e68cf11a None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:35.791125 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:35.834646 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7532ecb-56ed-4100-99aa-f35c0a7dd856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:35.835606 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7532ecb-56ed-4100-99aa-f35c0a7dd856 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:35.836146 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7532ecb-56ed-4100-99aa-f35c0a7dd856 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:35.836640 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7532ecb-56ed-4100-99aa-f35c0a7dd856 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:35.837513 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7532ecb-56ed-4100-99aa-f35c0a7dd856 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:35.839143 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7532ecb-56ed-4100-99aa-f35c0a7dd856 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:35.894331 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f65863-3e39-4ca6-a6f2-e61fd4e2bd4c None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:35.942120 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:35.959767 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 68/143] 10.222.0.27 () {60 vars in 1132 bytes} [Sun Sep 4 00:47:35 2022] POST /identity/v3/users => generated 283 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:47:36.051012 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f65863-3e39-4ca6-a6f2-e61fd4e2bd4c None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:36.101209 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52258c4-c87e-48bb-a801-a809f63e0949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:36.102224 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52258c4-c87e-48bb-a801-a809f63e0949 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:36.102705 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52258c4-c87e-48bb-a801-a809f63e0949 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:36.103175 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52258c4-c87e-48bb-a801-a809f63e0949 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:36.120967 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52258c4-c87e-48bb-a801-a809f63e0949 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:47:36.126467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-a52258c4-c87e-48bb-a801-a809f63e0949 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 00:47:36.126467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:36.126467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:36.126467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:36.126467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:36.126467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:36.126467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:36.126467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:36.126467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:36.126467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:36.126467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:36.126467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:36.126467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:47:36.126467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:47:36.126467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:47:36.126467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:47:36.127343 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:47:36.127343 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:47:36.127343 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:47:36.127343 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:47:36.127343 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:36.127343 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:36.127343 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:36.127343 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:36.127343 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:36.127343 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:36.127343 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:36.127343 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:36.127343 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:36.127343 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:36.127343 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:36.127343 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:36.128095 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:36.128095 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:36.128095 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:36.128095 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:36.128095 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:47:36.128095 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:47:36.128095 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:47:36.128095 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:47:36.128095 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:47:36.128095 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:47:36.128095 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 00:47:36.128095 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:47:36.131566 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 76/144] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:47:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:47:36.141657 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7532ecb-56ed-4100-99aa-f35c0a7dd856 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:36.261453 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89361e44-a23c-4e88-8500-af5f4eb0cbd6 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:36.261749 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89361e44-a23c-4e88-8500-af5f4eb0cbd6 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:36.262051 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89361e44-a23c-4e88-8500-af5f4eb0cbd6 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:36.264167 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 77/145] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:36.269229 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bae524-de57-42a2-8b70-6254d30ba2c6 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:36.269519 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bae524-de57-42a2-8b70-6254d30ba2c6 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:36.269795 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bae524-de57-42a2-8b70-6254d30ba2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:36.308421 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-57bae524-de57-42a2-8b70-6254d30ba2c6 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:36.309659 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-57bae524-de57-42a2-8b70-6254d30ba2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:36.326032 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7532ecb-56ed-4100-99aa-f35c0a7dd856 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:36.382612 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d6e80-a30a-498c-b18d-ede1f5662554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:36.383225 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d6e80-a30a-498c-b18d-ede1f5662554 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:36.383508 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d6e80-a30a-498c-b18d-ede1f5662554 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:36.383782 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d6e80-a30a-498c-b18d-ede1f5662554 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:36.384468 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266d6e80-a30a-498c-b18d-ede1f5662554 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:36.384787 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266d6e80-a30a-498c-b18d-ede1f5662554 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:36.386170 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266d6e80-a30a-498c-b18d-ede1f5662554 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:36.403390 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 69/146] 10.222.0.27 () {56 vars in 1101 bytes} [Sun Sep 4 00:47:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:36.409713 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d6e80-a30a-498c-b18d-ede1f5662554 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:36.450911 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 78/147] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 04 00:47:36.481017 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:36.539263 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d6e80-a30a-498c-b18d-ede1f5662554 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:36.578610 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6508fe59-2280-4448-9803-9ac65b70893b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:36.579243 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508fe59-2280-4448-9803-9ac65b70893b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:36.579527 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508fe59-2280-4448-9803-9ac65b70893b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:36.579804 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508fe59-2280-4448-9803-9ac65b70893b None admin] PATH_INFO: `/v3/users/5511ff4386ee4bb38c97add3954b95e0` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:36.581704 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6508fe59-2280-4448-9803-9ac65b70893b None admin] RBAC: Authorizing `identity:get_user(user_id=5511ff4386ee4bb38c97add3954b95e0)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:36.583147 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6508fe59-2280-4448-9803-9ac65b70893b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:36.585673 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 70/148] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:47:36 2022] GET /identity/v3/users/5511ff4386ee4bb38c97add3954b95e0 => generated 289 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:36.591448 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bae524-de57-42a2-8b70-6254d30ba2c6 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:36.628678 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:36.709281 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bae524-de57-42a2-8b70-6254d30ba2c6 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:36.749921 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa87ac-648e-498c-9d0f-3db888dfc34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:36.750563 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa87ac-648e-498c-9d0f-3db888dfc34f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:36.750841 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa87ac-648e-498c-9d0f-3db888dfc34f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:36.751116 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa87ac-648e-498c-9d0f-3db888dfc34f None admin] PATH_INFO: `/v3/projects/c8fe5139a2a24ec780814c7f39d5b437` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:36.752234 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa87ac-648e-498c-9d0f-3db888dfc34f None admin] RBAC: Authorizing `identity:get_project(project_id=c8fe5139a2a24ec780814c7f39d5b437)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:36.753731 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa87ac-648e-498c-9d0f-3db888dfc34f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:36.756182 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 79/149] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:47:36 2022] GET /identity/v3/projects/c8fe5139a2a24ec780814c7f39d5b437 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:36.761823 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6508fe59-2280-4448-9803-9ac65b70893b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:36.879474 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:36.892720 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6508fe59-2280-4448-9803-9ac65b70893b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:36.936177 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8899c25-c500-474a-8f4d-bf2b01cbec34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:36.936879 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8899c25-c500-474a-8f4d-bf2b01cbec34 None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:36.937164 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8899c25-c500-474a-8f4d-bf2b01cbec34 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:36.937448 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8899c25-c500-474a-8f4d-bf2b01cbec34 None admin] PATH_INFO: `/v3/projects/c8fe5139a2a24ec780814c7f39d5b437/users/5511ff4386ee4bb38c97add3954b95e0/roles/8f7a2ac695c6482cbf48819cee250c22` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:36.939466 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8899c25-c500-474a-8f4d-bf2b01cbec34 None admin] RBAC: Authorizing `identity:create_grant(project_id=c8fe5139a2a24ec780814c7f39d5b437, user_id=5511ff4386ee4bb38c97add3954b95e0, role_id=8f7a2ac695c6482cbf48819cee250c22)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:36.941067 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8899c25-c500-474a-8f4d-bf2b01cbec34 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:37.036989 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa87ac-648e-498c-9d0f-3db888dfc34f None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:37.071695 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 71/150] 10.222.0.27 () {58 vars in 1439 bytes} [Sun Sep 4 00:47:36 2022] PUT /identity/v3/projects/c8fe5139a2a24ec780814c7f39d5b437/users/5511ff4386ee4bb38c97add3954b95e0/roles/8f7a2ac695c6482cbf48819cee250c22 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 00:47:37.159469 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:37.164272 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa87ac-648e-498c-9d0f-3db888dfc34f None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:37.201667 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd237cc-c282-414d-b912-9a43848d5025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:37.202309 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd237cc-c282-414d-b912-9a43848d5025 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:37.202594 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd237cc-c282-414d-b912-9a43848d5025 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:37.202869 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd237cc-c282-414d-b912-9a43848d5025 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:37.217223 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd237cc-c282-414d-b912-9a43848d5025 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:47:37.225580 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-0dd237cc-c282-414d-b912-9a43848d5025 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:47:37.225580 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:37.225580 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:37.225580 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:37.225580 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:37.225580 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:37.225580 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:37.225580 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:37.225580 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:37.225580 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:37.225580 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:37.225580 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:37.225580 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:47:37.225580 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:47:37.225580 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:47:37.225580 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:47:37.227629 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:47:37.227629 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:47:37.227629 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:47:37.227629 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:47:37.227629 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:37.227629 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:37.227629 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:37.227629 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:37.227629 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:37.227629 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:37.227629 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:37.227629 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:37.227629 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:37.227629 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:37.227629 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:37.227629 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:37.229497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:37.229497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:37.229497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:37.229497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:37.229497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:47:37.229497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:47:37.229497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:47:37.229497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:47:37.229497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:47:37.229497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:47:37.229497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:47:37.229497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:47:37.229497 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 80/151] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:47:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:47:37.233971 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8899c25-c500-474a-8f4d-bf2b01cbec34 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:37.377941 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8899c25-c500-474a-8f4d-bf2b01cbec34 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:37.419993 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f5064-da02-4d9c-aec0-bb7c18336a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:37.420655 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f5064-da02-4d9c-aec0-bb7c18336a6b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:37.420939 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f5064-da02-4d9c-aec0-bb7c18336a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:37.421216 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f5064-da02-4d9c-aec0-bb7c18336a6b None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:37.421886 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578f5064-da02-4d9c-aec0-bb7c18336a6b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:37.422338 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578f5064-da02-4d9c-aec0-bb7c18336a6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:37.423734 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578f5064-da02-4d9c-aec0-bb7c18336a6b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:37.436576 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 72/152] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:47:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:37.441860 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd237cc-c282-414d-b912-9a43848d5025 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:37.541362 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d3f19-1480-4f8e-b8e3-6296aed917ca None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:37.541691 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d3f19-1480-4f8e-b8e3-6296aed917ca None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:37.541907 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d3f19-1480-4f8e-b8e3-6296aed917ca None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:37.543794 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 73/153] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:37.549284 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed763b-91a7-4280-9c70-14027cbb3aee None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:37.549571 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed763b-91a7-4280-9c70-14027cbb3aee None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:37.549853 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed763b-91a7-4280-9c70-14027cbb3aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:37.576819 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd237cc-c282-414d-b912-9a43848d5025 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:37.593711 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-9bed763b-91a7-4280-9c70-14027cbb3aee None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:37.594967 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9bed763b-91a7-4280-9c70-14027cbb3aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:37.599583 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:37.624987 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d843ca4d-41e2-486d-9330-243a0f81f28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:37.625917 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d843ca4d-41e2-486d-9330-243a0f81f28a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:37.626207 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d843ca4d-41e2-486d-9330-243a0f81f28a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:37.626499 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d843ca4d-41e2-486d-9330-243a0f81f28a None admin] PATH_INFO: `/v3/users/644f0b1b120742d2aca15aeeaa76abcc` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:37.627665 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d843ca4d-41e2-486d-9330-243a0f81f28a None admin] RBAC: Authorizing `identity:get_user(user_id=644f0b1b120742d2aca15aeeaa76abcc)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:37.629226 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d843ca4d-41e2-486d-9330-243a0f81f28a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:37.632527 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 81/154] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:47:37 2022] GET /identity/v3/users/644f0b1b120742d2aca15aeeaa76abcc => generated 247 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:37.639915 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d843ca4d-41e2-486d-9330-243a0f81f28a None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:37.678678 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:37.730257 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 74/155] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:37.775883 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d843ca4d-41e2-486d-9330-243a0f81f28a None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:37.820912 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed7385-ec0e-478b-a0b9-df5272792d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:37.821526 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed7385-ec0e-478b-a0b9-df5272792d43 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:37.821805 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed7385-ec0e-478b-a0b9-df5272792d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:37.822075 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed7385-ec0e-478b-a0b9-df5272792d43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:37.899692 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ed7385-ec0e-478b-a0b9-df5272792d43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:37.901143 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ed7385-ec0e-478b-a0b9-df5272792d43 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:37.934978 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:38.029490 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 82/156] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:47:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:47:38.035392 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bed763b-91a7-4280-9c70-14027cbb3aee None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:38.205252 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bed763b-91a7-4280-9c70-14027cbb3aee None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:38.259554 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49025617-2bea-4655-897a-315aa2cd7b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:38.260293 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49025617-2bea-4655-897a-315aa2cd7b3e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:38.260500 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49025617-2bea-4655-897a-315aa2cd7b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:38.260777 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49025617-2bea-4655-897a-315aa2cd7b3e None admin] PATH_INFO: `/v3/projects/c8fe5139a2a24ec780814c7f39d5b437` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:38.265148 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49025617-2bea-4655-897a-315aa2cd7b3e None admin] RBAC: Authorizing `identity:get_project(project_id=c8fe5139a2a24ec780814c7f39d5b437)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:38.266813 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49025617-2bea-4655-897a-315aa2cd7b3e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:38.268831 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 75/157] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:47:38 2022] GET /identity/v3/projects/c8fe5139a2a24ec780814c7f39d5b437 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:38.275372 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed7385-ec0e-478b-a0b9-df5272792d43 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:38.353597 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49025617-2bea-4655-897a-315aa2cd7b3e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:38.374965 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:38.509291 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed7385-ec0e-478b-a0b9-df5272792d43 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:38.556843 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:38.563902 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73874176-848e-46a1-b2f8-e4309fe7636e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:38.564681 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73874176-848e-46a1-b2f8-e4309fe7636e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:38.564959 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73874176-848e-46a1-b2f8-e4309fe7636e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:38.565352 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73874176-848e-46a1-b2f8-e4309fe7636e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:38.566213 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73874176-848e-46a1-b2f8-e4309fe7636e None admin] RBAC: Adding query filter params (role.id=c8a59d34ee2f4cefa6940cac778260c7, scope.project.id=c8fe5139a2a24ec780814c7f39d5b437, user.id=644f0b1b120742d2aca15aeeaa76abcc) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:38.571726 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73874176-848e-46a1-b2f8-e4309fe7636e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:38.571726 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73874176-848e-46a1-b2f8-e4309fe7636e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:38.580929 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 83/158] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:47:38 2022] GET /identity/v3/role_assignments?user.id=644f0b1b120742d2aca15aeeaa76abcc&scope.project.id=c8fe5139a2a24ec780814c7f39d5b437&role.id=c8a59d34ee2f4cefa6940cac778260c7 => generated 608 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:38.590819 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49025617-2bea-4655-897a-315aa2cd7b3e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:38.596468 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064d923-65ce-4686-a515-f100f974c672 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:38.596775 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064d923-65ce-4686-a515-f100f974c672 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:38.597040 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064d923-65ce-4686-a515-f100f974c672 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:38.598484 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 84/159] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 00:47:38.605406 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729bf98-bda7-4345-b10f-1b9e3926d3b9 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:38.605805 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729bf98-bda7-4345-b10f-1b9e3926d3b9 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:38.606096 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729bf98-bda7-4345-b10f-1b9e3926d3b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:38.643690 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-9729bf98-bda7-4345-b10f-1b9e3926d3b9 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:38.644936 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-9729bf98-bda7-4345-b10f-1b9e3926d3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:38.677860 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399948e-f892-4bc4-9779-a168ed689e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:38.678522 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399948e-f892-4bc4-9779-a168ed689e74 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:38.678806 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399948e-f892-4bc4-9779-a168ed689e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:38.679077 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399948e-f892-4bc4-9779-a168ed689e74 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:38.704957 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e399948e-f892-4bc4-9779-a168ed689e74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:47:38.711002 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-e399948e-f892-4bc4-9779-a168ed689e74 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:47:38.711002 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:38.711002 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:38.711002 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:38.711002 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:38.711002 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:38.711002 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:38.711002 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:38.711002 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:38.711002 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:38.711002 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:38.711002 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:38.711002 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:47:38.711002 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:47:38.711002 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:47:38.711002 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:47:38.711886 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:47:38.711886 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:47:38.711886 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:47:38.711886 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:47:38.711886 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:38.711886 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:38.711886 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:38.711886 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:38.711886 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:38.711886 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:38.711886 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:38.711886 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:38.711886 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:38.711886 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:38.711886 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:38.711886 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:38.712737 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:38.712737 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:38.712737 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:38.712737 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:38.712737 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:47:38.712737 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:47:38.712737 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:47:38.712737 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:47:38.712737 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:47:38.712737 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:47:38.712737 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:47:38.712737 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:47:38.712737 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 76/160] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:47:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:47:38.724815 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5baa40-d3c9-476e-a80d-83bae3291b01 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:38.725107 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5baa40-d3c9-476e-a80d-83bae3291b01 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:38.725384 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5baa40-d3c9-476e-a80d-83bae3291b01 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:38.726727 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 77/161] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:38 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:38.727485 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5baa40-d3c9-476e-a80d-83bae3291b01 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:38.809450 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 85/162] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:38.810658 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9caffc5-0821-4c3b-a080-ff4ec555a648 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:38.810940 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9caffc5-0821-4c3b-a080-ff4ec555a648 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:38.811211 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9caffc5-0821-4c3b-a080-ff4ec555a648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:38.862607 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-c9caffc5-0821-4c3b-a080-ff4ec555a648 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:38.863831 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c9caffc5-0821-4c3b-a080-ff4ec555a648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:38.908514 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:39.001297 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5baa40-d3c9-476e-a80d-83bae3291b01 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:39.015143 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 86/163] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:39.053953 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b6629-6052-455b-b07d-d180dc027419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:39.054572 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b6629-6052-455b-b07d-d180dc027419 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:39.054852 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b6629-6052-455b-b07d-d180dc027419 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:39.055125 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b6629-6052-455b-b07d-d180dc027419 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:39.055795 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b6629-6052-455b-b07d-d180dc027419 None admin] RBAC: Adding query filter params (name=member) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:39.056093 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b6629-6052-455b-b07d-d180dc027419 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:39.057511 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b6629-6052-455b-b07d-d180dc027419 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:39.071792 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 78/164] 10.222.0.27 () {56 vars in 1101 bytes} [Sun Sep 4 00:47:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:39.077741 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9caffc5-0821-4c3b-a080-ff4ec555a648 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:39.139306 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:39.199623 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9caffc5-0821-4c3b-a080-ff4ec555a648 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:39.263328 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd8531-34a0-4e32-9210-e5c4b390d1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:39.263968 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd8531-34a0-4e32-9210-e5c4b390d1e5 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:39.264250 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd8531-34a0-4e32-9210-e5c4b390d1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:39.264561 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd8531-34a0-4e32-9210-e5c4b390d1e5 None admin] PATH_INFO: `/v3/users/43855050fbcd47d3b5ded0e8ac79e20d` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:39.266516 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd8531-34a0-4e32-9210-e5c4b390d1e5 None admin] RBAC: Authorizing `identity:get_user(user_id=43855050fbcd47d3b5ded0e8ac79e20d)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:39.268376 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd8531-34a0-4e32-9210-e5c4b390d1e5 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:39.269992 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 87/165] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:47:39 2022] GET /identity/v3/users/43855050fbcd47d3b5ded0e8ac79e20d => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:39.276343 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b6629-6052-455b-b07d-d180dc027419 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:39.393928 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b6629-6052-455b-b07d-d180dc027419 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:39.412232 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd8531-34a0-4e32-9210-e5c4b390d1e5 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:39.452312 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:39.462742 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4943d86-6062-4af7-91f5-80b25c618f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:39.469592 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4943d86-6062-4af7-91f5-80b25c618f0e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:39.469592 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4943d86-6062-4af7-91f5-80b25c618f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:39.469592 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4943d86-6062-4af7-91f5-80b25c618f0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:39.562585 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4943d86-6062-4af7-91f5-80b25c618f0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:39.563474 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd8531-34a0-4e32-9210-e5c4b390d1e5 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:39.564132 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4943d86-6062-4af7-91f5-80b25c618f0e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:39.605369 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:39.641743 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52d4ef-146f-404c-b826-87cf21c2d34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:39.642398 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52d4ef-146f-404c-b826-87cf21c2d34a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:39.642671 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52d4ef-146f-404c-b826-87cf21c2d34a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:39.642932 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52d4ef-146f-404c-b826-87cf21c2d34a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:39.666512 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa52d4ef-146f-404c-b826-87cf21c2d34a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:47:39.687885 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-aa52d4ef-146f-404c-b826-87cf21c2d34a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:47:39.687885 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:39.687885 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:39.687885 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:39.687885 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:39.687885 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:39.687885 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:39.687885 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:39.687885 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:39.687885 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:39.687885 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:39.687885 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:39.687885 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:47:39.687885 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:47:39.687885 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:47:39.687885 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:47:39.688861 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:47:39.688861 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:47:39.688861 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:47:39.688861 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:47:39.688861 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:39.688861 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:39.688861 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:39.688861 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:39.688861 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:39.688861 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:39.688861 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:39.688861 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:39.688861 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:39.688861 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:39.688861 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:39.688861 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:39.690192 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:39.690192 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:39.690192 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:39.690192 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:39.690192 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:47:39.690192 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:47:39.690192 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:47:39.690192 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:47:39.690192 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:47:39.690192 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:47:39.690192 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:47:39.690192 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:47:39.690192 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 88/166] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:47:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:47:39.690192 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52d4ef-146f-404c-b826-87cf21c2d34a None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:39.700984 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 79/167] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:47:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:47:39.701904 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4943d86-6062-4af7-91f5-80b25c618f0e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:39.771377 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:39.894547 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4943d86-6062-4af7-91f5-80b25c618f0e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:39.899165 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52d4ef-146f-404c-b826-87cf21c2d34a None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:39.958870 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281bc25-4a13-479f-8107-f8b5f9fced4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:39.959712 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ce4de-b366-4dd0-8a78-b05129d393fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:39.960327 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ce4de-b366-4dd0-8a78-b05129d393fb None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:39.960628 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ce4de-b366-4dd0-8a78-b05129d393fb None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:39.960898 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ce4de-b366-4dd0-8a78-b05129d393fb None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:39.961575 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ce4de-b366-4dd0-8a78-b05129d393fb None admin] RBAC: Adding query filter params (name=member) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:39.961874 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ce4de-b366-4dd0-8a78-b05129d393fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:39.963268 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ce4de-b366-4dd0-8a78-b05129d393fb None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:39.964927 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281bc25-4a13-479f-8107-f8b5f9fced4a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:39.965204 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281bc25-4a13-479f-8107-f8b5f9fced4a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:39.965472 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281bc25-4a13-479f-8107-f8b5f9fced4a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:39.984904 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0281bc25-4a13-479f-8107-f8b5f9fced4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:47:39.987354 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 80/168] 10.222.0.27 () {56 vars in 1101 bytes} [Sun Sep 4 00:47:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:39.988217 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ce4de-b366-4dd0-8a78-b05129d393fb None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:39.996772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-0281bc25-4a13-479f-8107-f8b5f9fced4a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 00:47:39.996772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:39.996772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:39.996772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:39.996772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:39.996772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:39.996772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:39.996772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:39.996772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:39.996772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:39.996772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:39.996772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:39.996772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:47:39.996772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:47:39.996772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:47:39.996772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:47:39.997651 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:47:39.997651 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:47:39.997651 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:47:39.997651 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:47:39.997651 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:39.997651 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:39.997651 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:39.997651 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:39.997651 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:39.997651 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:39.997651 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:39.997651 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:39.997651 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:39.997651 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:39.997651 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:39.997651 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:39.998766 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:39.998766 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:39.998766 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:39.998766 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:39.998766 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:47:39.998766 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:47:39.998766 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:47:39.998766 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:47:39.998766 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:47:39.998766 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:47:39.998766 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 00:47:39.998766 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:47:39.998766 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 89/169] 10.222.0.27 () {56 vars in 1114 bytes} [Sun Sep 4 00:47:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:47:39.998766 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281bc25-4a13-479f-8107-f8b5f9fced4a None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:40.080846 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:40.160797 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:40.175142 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ce4de-b366-4dd0-8a78-b05129d393fb None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:40.184770 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281bc25-4a13-479f-8107-f8b5f9fced4a None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:40.245741 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b59be4-8782-41cd-b0a6-bdf3c5fa6f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:40.246373 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b59be4-8782-41cd-b0a6-bdf3c5fa6f5a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:40.246647 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b59be4-8782-41cd-b0a6-bdf3c5fa6f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:40.246917 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b59be4-8782-41cd-b0a6-bdf3c5fa6f5a None admin] PATH_INFO: `/v3/projects/c8fe5139a2a24ec780814c7f39d5b437` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:40.248700 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b59be4-8782-41cd-b0a6-bdf3c5fa6f5a None admin] RBAC: Authorizing `identity:get_project(project_id=c8fe5139a2a24ec780814c7f39d5b437)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:40.250140 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b59be4-8782-41cd-b0a6-bdf3c5fa6f5a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:40.251761 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 81/170] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:47:39 2022] GET /identity/v3/projects/c8fe5139a2a24ec780814c7f39d5b437 => generated 298 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:40.257207 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b59be4-8782-41cd-b0a6-bdf3c5fa6f5a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:40.262810 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d4266c-d756-49a9-86fc-2da60c71e465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:40.263451 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4266c-d756-49a9-86fc-2da60c71e465 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:40.263727 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4266c-d756-49a9-86fc-2da60c71e465 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:40.264005 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4266c-d756-49a9-86fc-2da60c71e465 None admin] PATH_INFO: `/v3/users/43855050fbcd47d3b5ded0e8ac79e20d` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:40.266458 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d4266c-d756-49a9-86fc-2da60c71e465 None admin] RBAC: Authorizing `identity:get_user(user_id=43855050fbcd47d3b5ded0e8ac79e20d)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:40.268777 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d4266c-d756-49a9-86fc-2da60c71e465 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:40.270377 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 90/171] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:47:39 2022] GET /identity/v3/users/43855050fbcd47d3b5ded0e8ac79e20d => generated 275 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:40.271182 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d4266c-d756-49a9-86fc-2da60c71e465 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:40.407431 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b59be4-8782-41cd-b0a6-bdf3c5fa6f5a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:40.467800 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab11b1a-b64b-48f8-959d-269fe522b003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:40.468413 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab11b1a-b64b-48f8-959d-269fe522b003 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:40.468720 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab11b1a-b64b-48f8-959d-269fe522b003 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:40.468991 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab11b1a-b64b-48f8-959d-269fe522b003 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:40.469659 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab11b1a-b64b-48f8-959d-269fe522b003 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:40.469956 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab11b1a-b64b-48f8-959d-269fe522b003 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:40.471335 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab11b1a-b64b-48f8-959d-269fe522b003 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:40.488770 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 82/172] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:47:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:40.497297 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab11b1a-b64b-48f8-959d-269fe522b003 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:40.522849 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d4266c-d756-49a9-86fc-2da60c71e465 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:40.537270 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:40.550721 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:40.592631 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffe9d4f-dd3a-4418-82ac-e6f078549ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:40.593287 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe9d4f-dd3a-4418-82ac-e6f078549ac1 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:40.593574 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe9d4f-dd3a-4418-82ac-e6f078549ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:40.593843 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe9d4f-dd3a-4418-82ac-e6f078549ac1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:40.594583 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffe9d4f-dd3a-4418-82ac-e6f078549ac1 None admin] RBAC: Adding query filter params (role.id=d1261c99e1d64f9da63adfff739561e8, scope.project.id=c8fe5139a2a24ec780814c7f39d5b437, user.id=43855050fbcd47d3b5ded0e8ac79e20d) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:40.594888 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffe9d4f-dd3a-4418-82ac-e6f078549ac1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:40.596344 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffe9d4f-dd3a-4418-82ac-e6f078549ac1 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:40.610681 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 91/173] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:47:40 2022] GET /identity/v3/role_assignments?user.id=43855050fbcd47d3b5ded0e8ac79e20d&scope.project.id=c8fe5139a2a24ec780814c7f39d5b437&role.id=d1261c99e1d64f9da63adfff739561e8 => generated 608 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:40.612023 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ca0fa-6597-4c41-aaf7-c56b7a0f60ef None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:40.612308 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ca0fa-6597-4c41-aaf7-c56b7a0f60ef None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:40.612649 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ca0fa-6597-4c41-aaf7-c56b7a0f60ef None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:40.613938 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 92/174] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:40.614554 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ca0fa-6597-4c41-aaf7-c56b7a0f60ef None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:40.698877 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab11b1a-b64b-48f8-959d-269fe522b003 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:40.774263 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0618110-1ea2-4083-84fa-055b65906954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:40.774892 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0618110-1ea2-4083-84fa-055b65906954 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:40.775173 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0618110-1ea2-4083-84fa-055b65906954 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:40.775449 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0618110-1ea2-4083-84fa-055b65906954 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:40.820300 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ca0fa-6597-4c41-aaf7-c56b7a0f60ef None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:40.863650 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64aec60-adcb-4c30-b6a2-5a5d25eb9a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:40.865028 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64aec60-adcb-4c30-b6a2-5a5d25eb9a51 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:40.865416 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64aec60-adcb-4c30-b6a2-5a5d25eb9a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:40.865687 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64aec60-adcb-4c30-b6a2-5a5d25eb9a51 None admin] PATH_INFO: `/v3/users/43855050fbcd47d3b5ded0e8ac79e20d` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:40.866833 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64aec60-adcb-4c30-b6a2-5a5d25eb9a51 None admin] RBAC: Authorizing `identity:get_user(user_id=43855050fbcd47d3b5ded0e8ac79e20d)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:40.868298 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64aec60-adcb-4c30-b6a2-5a5d25eb9a51 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:40.869397 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0618110-1ea2-4083-84fa-055b65906954 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:40.870872 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0618110-1ea2-4083-84fa-055b65906954 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:40.871907 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 93/175] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:47:40 2022] GET /identity/v3/users/43855050fbcd47d3b5ded0e8ac79e20d => generated 275 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:40.872945 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d84b4-fba4-4202-b5a6-eb78ddbc122c None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:40.873226 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d84b4-fba4-4202-b5a6-eb78ddbc122c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:40.873496 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d84b4-fba4-4202-b5a6-eb78ddbc122c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:40.896855 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-ed6d84b4-fba4-4202-b5a6-eb78ddbc122c None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:40.898288 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ed6d84b4-fba4-4202-b5a6-eb78ddbc122c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:40.899481 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:40.933465 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:40.971638 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 94/176] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:40.972990 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915d6f8-1e64-45dd-b2fe-07c80d1e52d0 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:40.973280 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915d6f8-1e64-45dd-b2fe-07c80d1e52d0 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:40.973611 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915d6f8-1e64-45dd-b2fe-07c80d1e52d0 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:40.975921 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 95/177] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:40.977107 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4272178-f349-48fe-ae12-918aeaabfeb3 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:40.977393 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4272178-f349-48fe-ae12-918aeaabfeb3 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:40.977668 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4272178-f349-48fe-ae12-918aeaabfeb3 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:40.980495 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 96/178] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:40.980495 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4272178-f349-48fe-ae12-918aeaabfeb3 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:40.987519 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 83/179] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:47:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:47:40.988907 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29562f81-0c83-499c-8586-9775c1be90a5 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:40.989228 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29562f81-0c83-499c-8586-9775c1be90a5 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:40.989513 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29562f81-0c83-499c-8586-9775c1be90a5 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:40.990925 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 84/180] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:40.991771 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ebcb9-917e-44a9-817e-28b1c00214aa None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:40.992047 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ebcb9-917e-44a9-817e-28b1c00214aa None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:40.992317 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ebcb9-917e-44a9-817e-28b1c00214aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:41.009804 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-b31ebcb9-917e-44a9-817e-28b1c00214aa None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:41.010990 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b31ebcb9-917e-44a9-817e-28b1c00214aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:41.084162 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 85/181] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:41.086764 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c21a25-942a-4359-9eca-56c24c534177 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:41.087233 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c21a25-942a-4359-9eca-56c24c534177 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:41.087959 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c21a25-942a-4359-9eca-56c24c534177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:41.106288 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4272178-f349-48fe-ae12-918aeaabfeb3 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:41.112120 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-44c21a25-942a-4359-9eca-56c24c534177 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:41.113553 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-44c21a25-942a-4359-9eca-56c24c534177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:41.142487 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23297aed-d615-4729-8643-d7ad5b91257d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:41.143172 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23297aed-d615-4729-8643-d7ad5b91257d None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:41.143518 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23297aed-d615-4729-8643-d7ad5b91257d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:41.143876 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23297aed-d615-4729-8643-d7ad5b91257d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:41.181161 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:41.182647 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:41.203886 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 86/182] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:41.212927 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a594f6c-b0b4-4991-9a08-d5489d5f0c3a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:41.213424 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a594f6c-b0b4-4991-9a08-d5489d5f0c3a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:41.213899 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a594f6c-b0b4-4991-9a08-d5489d5f0c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:41.215821 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23297aed-d615-4729-8643-d7ad5b91257d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:41.217328 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23297aed-d615-4729-8643-d7ad5b91257d None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:41.243614 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5a594f6c-b0b4-4991-9a08-d5489d5f0c3a None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:41.245853 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5a594f6c-b0b4-4991-9a08-d5489d5f0c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:41.323537 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 87/183] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:41.324211 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a594f6c-b0b4-4991-9a08-d5489d5f0c3a None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:41.329598 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 97/184] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:47:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:47:41.330712 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23297aed-d615-4729-8643-d7ad5b91257d None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:41.416272 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:41.434260 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a594f6c-b0b4-4991-9a08-d5489d5f0c3a None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:41.447512 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23297aed-d615-4729-8643-d7ad5b91257d None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:41.448371 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:41.470328 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f5a40-7e95-4d1c-8f11-cc1f5df47d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:41.471201 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f5a40-7e95-4d1c-8f11-cc1f5df47d6a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:41.471622 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f5a40-7e95-4d1c-8f11-cc1f5df47d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:41.472034 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f5a40-7e95-4d1c-8f11-cc1f5df47d6a None admin] PATH_INFO: `/v3/projects/54786471df9a4027b37cc1f2e82d05e6` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:41.473347 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f5a40-7e95-4d1c-8f11-cc1f5df47d6a None admin] RBAC: Authorizing `identity:get_project(project_id=54786471df9a4027b37cc1f2e82d05e6)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:41.475927 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f5a40-7e95-4d1c-8f11-cc1f5df47d6a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:41.477739 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 88/185] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:47:41 2022] GET /identity/v3/projects/54786471df9a4027b37cc1f2e82d05e6 => generated 312 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:41.478638 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f5a40-7e95-4d1c-8f11-cc1f5df47d6a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:41.487503 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13fd0a2-b454-42e3-97f2-0039b267d6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:41.488144 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fd0a2-b454-42e3-97f2-0039b267d6ed None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:41.488422 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fd0a2-b454-42e3-97f2-0039b267d6ed None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:41.488731 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fd0a2-b454-42e3-97f2-0039b267d6ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:41.500750 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13fd0a2-b454-42e3-97f2-0039b267d6ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:47:41.505873 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-d13fd0a2-b454-42e3-97f2-0039b267d6ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:47:41.505873 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:41.505873 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:41.505873 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:41.505873 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:41.505873 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:41.505873 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:41.505873 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:41.505873 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:41.505873 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:41.505873 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:41.505873 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:41.505873 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:47:41.505873 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:47:41.505873 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:47:41.505873 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:47:41.506923 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:47:41.506923 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:47:41.506923 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:47:41.506923 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:47:41.506923 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:41.506923 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:41.506923 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:41.506923 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:41.506923 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:41.506923 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:41.506923 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:41.506923 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:41.506923 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:41.506923 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:41.506923 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:41.506923 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:41.507826 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:41.507826 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:41.507826 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:41.507826 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:41.507826 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:47:41.507826 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:47:41.507826 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:47:41.507826 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:47:41.507826 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:47:41.507826 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:47:41.507826 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:47:41.507826 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:47:41.508435 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 98/186] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:47:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:47:41.513016 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13fd0a2-b454-42e3-97f2-0039b267d6ed None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:41.593857 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f5a40-7e95-4d1c-8f11-cc1f5df47d6a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:41.626494 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13fd0a2-b454-42e3-97f2-0039b267d6ed None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:41.628549 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300ed85-dadd-4955-b95e-f0c0979da20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:41.629188 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300ed85-dadd-4955-b95e-f0c0979da20f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:41.629471 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300ed85-dadd-4955-b95e-f0c0979da20f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:41.629815 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300ed85-dadd-4955-b95e-f0c0979da20f None admin] PATH_INFO: `/v3/projects/c8fe5139a2a24ec780814c7f39d5b437` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:41.631032 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e300ed85-dadd-4955-b95e-f0c0979da20f None admin] RBAC: Authorizing `identity:get_project(project_id=c8fe5139a2a24ec780814c7f39d5b437)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:41.632496 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e300ed85-dadd-4955-b95e-f0c0979da20f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:41.635146 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 89/187] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:47:41 2022] GET /identity/v3/projects/c8fe5139a2a24ec780814c7f39d5b437 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:41.635146 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300ed85-dadd-4955-b95e-f0c0979da20f None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:41.655487 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:41.663315 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804dc912-425b-484a-98a2-2304071908fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:41.663940 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804dc912-425b-484a-98a2-2304071908fa None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:41.664277 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804dc912-425b-484a-98a2-2304071908fa None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:41.664583 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804dc912-425b-484a-98a2-2304071908fa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:41.675439 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804dc912-425b-484a-98a2-2304071908fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:47:41.681220 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-804dc912-425b-484a-98a2-2304071908fa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 00:47:41.681220 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:41.681220 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:41.681220 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:41.681220 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:41.681220 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:41.681220 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:41.681220 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:41.681220 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:41.681220 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:41.681220 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:41.681220 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:41.681220 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:47:41.681220 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:47:41.681220 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:47:41.681220 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:47:41.682093 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:47:41.682093 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:47:41.682093 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:47:41.682093 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:47:41.682093 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:41.682093 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:41.682093 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:41.682093 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:41.682093 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:41.682093 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:41.682093 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:41.682093 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:41.682093 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:41.682093 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:41.682093 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:41.682093 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:41.682850 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:41.682850 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:41.682850 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:41.682850 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:41.682850 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:47:41.682850 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:47:41.682850 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:47:41.682850 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:47:41.682850 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:47:41.682850 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:47:41.682850 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 00:47:41.682850 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:47:41.682850 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 99/188] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:47:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:47:41.685179 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:41.685389 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804dc912-425b-484a-98a2-2304071908fa None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:41.735933 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300ed85-dadd-4955-b95e-f0c0979da20f None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:41.768436 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03cbd4-1ad2-4e93-bd13-137ce068fabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:41.771764 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03cbd4-1ad2-4e93-bd13-137ce068fabf None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:41.772044 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03cbd4-1ad2-4e93-bd13-137ce068fabf None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:41.772314 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03cbd4-1ad2-4e93-bd13-137ce068fabf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:41.773057 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d03cbd4-1ad2-4e93-bd13-137ce068fabf None admin] RBAC: Adding query filter params (role.id=d1261c99e1d64f9da63adfff739561e8, scope.project.id=54786471df9a4027b37cc1f2e82d05e6, user.id=43855050fbcd47d3b5ded0e8ac79e20d) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:41.773363 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d03cbd4-1ad2-4e93-bd13-137ce068fabf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:41.774779 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d03cbd4-1ad2-4e93-bd13-137ce068fabf None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:41.780500 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 90/189] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:47:41 2022] GET /identity/v3/role_assignments?user.id=43855050fbcd47d3b5ded0e8ac79e20d&scope.project.id=54786471df9a4027b37cc1f2e82d05e6&role.id=d1261c99e1d64f9da63adfff739561e8 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:41.781209 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03cbd4-1ad2-4e93-bd13-137ce068fabf None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:41.784278 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804dc912-425b-484a-98a2-2304071908fa None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:41.825799 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45339bea-fd4b-4690-b29e-ed28dc44beca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:41.826492 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45339bea-fd4b-4690-b29e-ed28dc44beca None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:41.826795 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45339bea-fd4b-4690-b29e-ed28dc44beca None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:41.827088 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45339bea-fd4b-4690-b29e-ed28dc44beca None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:41.827822 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45339bea-fd4b-4690-b29e-ed28dc44beca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:41.828126 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45339bea-fd4b-4690-b29e-ed28dc44beca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:41.829668 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45339bea-fd4b-4690-b29e-ed28dc44beca None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:41.840899 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 100/190] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:47:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:41.842104 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45339bea-fd4b-4690-b29e-ed28dc44beca None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:41.899473 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:41.905237 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03cbd4-1ad2-4e93-bd13-137ce068fabf None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:41.920181 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:41.948375 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472cf347-7023-48ee-91d2-a9acf40987eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:41.949281 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472cf347-7023-48ee-91d2-a9acf40987eb None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:41.949723 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472cf347-7023-48ee-91d2-a9acf40987eb None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:41.950135 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472cf347-7023-48ee-91d2-a9acf40987eb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:41.954203 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45339bea-fd4b-4690-b29e-ed28dc44beca None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:41.963305 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472cf347-7023-48ee-91d2-a9acf40987eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:47:41.968724 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-472cf347-7023-48ee-91d2-a9acf40987eb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 00:47:41.968724 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:41.968724 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:41.968724 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:41.968724 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:41.968724 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:41.968724 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:41.968724 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:41.968724 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:41.968724 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:41.968724 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:41.968724 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:41.968724 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:47:41.968724 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:47:41.968724 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:47:41.968724 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:47:41.969643 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:47:41.969643 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:47:41.969643 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:47:41.969643 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:47:41.969643 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:41.969643 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:41.969643 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:41.969643 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:41.969643 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:41.969643 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:41.969643 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:41.969643 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:41.969643 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:41.969643 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:41.969643 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:41.969643 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:41.970469 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:41.970469 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:41.970469 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:41.970469 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:41.970469 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:47:41.970469 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:47:41.970469 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:47:41.970469 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:47:41.970469 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:47:41.970469 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:47:41.970469 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 00:47:41.970469 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:47:41.970469 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 91/191] 10.222.0.27 () {56 vars in 1114 bytes} [Sun Sep 4 00:47:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:47:41.971146 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472cf347-7023-48ee-91d2-a9acf40987eb None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:41.992923 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a02e71-52d4-42f5-826d-a442d99832cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:41.993358 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a02e71-52d4-42f5-826d-a442d99832cd None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:41.993639 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a02e71-52d4-42f5-826d-a442d99832cd None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:41.993904 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a02e71-52d4-42f5-826d-a442d99832cd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:41.995041 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a02e71-52d4-42f5-826d-a442d99832cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:41.996538 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a02e71-52d4-42f5-826d-a442d99832cd None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:41.998313 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 101/192] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:47:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:41.998911 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a02e71-52d4-42f5-826d-a442d99832cd None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:42.075079 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472cf347-7023-48ee-91d2-a9acf40987eb None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:42.096700 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a02e71-52d4-42f5-826d-a442d99832cd None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:42.108312 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af403e4-44c2-4bf5-8983-ad40b4f2accf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:42.108938 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af403e4-44c2-4bf5-8983-ad40b4f2accf None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:42.109218 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af403e4-44c2-4bf5-8983-ad40b4f2accf None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:42.109491 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af403e4-44c2-4bf5-8983-ad40b4f2accf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:42.110213 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af403e4-44c2-4bf5-8983-ad40b4f2accf None admin] RBAC: Adding query filter params (role.id=7df9ebc56a2a4214b2b64a5941812ef0, scope.project.id=c8fe5139a2a24ec780814c7f39d5b437, user.id=43855050fbcd47d3b5ded0e8ac79e20d) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:42.110521 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af403e4-44c2-4bf5-8983-ad40b4f2accf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:42.111963 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af403e4-44c2-4bf5-8983-ad40b4f2accf None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:42.115992 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:42.118499 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 92/193] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:47:41 2022] GET /identity/v3/role_assignments?user.id=43855050fbcd47d3b5ded0e8ac79e20d&scope.project.id=c8fe5139a2a24ec780814c7f39d5b437&role.id=7df9ebc56a2a4214b2b64a5941812ef0 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:42.132724 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af403e4-44c2-4bf5-8983-ad40b4f2accf None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:42.133243 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7b0ec-bba8-4eec-8353-7529e038a499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:42.133875 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7b0ec-bba8-4eec-8353-7529e038a499 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:42.134155 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7b0ec-bba8-4eec-8353-7529e038a499 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:42.134426 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7b0ec-bba8-4eec-8353-7529e038a499 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:42.135113 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f7b0ec-bba8-4eec-8353-7529e038a499 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:42.135416 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f7b0ec-bba8-4eec-8353-7529e038a499 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:42.137007 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f7b0ec-bba8-4eec-8353-7529e038a499 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:42.146820 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 102/194] 10.222.0.27 () {56 vars in 1101 bytes} [Sun Sep 4 00:47:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:42.147517 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7b0ec-bba8-4eec-8353-7529e038a499 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:42.169987 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:42.250168 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af403e4-44c2-4bf5-8983-ad40b4f2accf None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:42.274104 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7b0ec-bba8-4eec-8353-7529e038a499 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:42.291370 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b265c214-a144-4d51-831c-df318274956e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:42.292298 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265c214-a144-4d51-831c-df318274956e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:42.292693 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265c214-a144-4d51-831c-df318274956e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:42.292976 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265c214-a144-4d51-831c-df318274956e None admin] PATH_INFO: `/v3/users/9c6861d39c0949349fb24061ef1c8249` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:42.309717 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b265c214-a144-4d51-831c-df318274956e None admin] RBAC: Authorizing `identity:get_user(user_id=9c6861d39c0949349fb24061ef1c8249)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:42.312210 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b265c214-a144-4d51-831c-df318274956e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:42.313497 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 93/195] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:47:42 2022] GET /identity/v3/users/9c6861d39c0949349fb24061ef1c8249 => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:42.314077 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b265c214-a144-4d51-831c-df318274956e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:42.314941 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887220ac-b56b-4e50-af60-3d0be47bebbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:42.315553 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887220ac-b56b-4e50-af60-3d0be47bebbf None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:42.315830 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887220ac-b56b-4e50-af60-3d0be47bebbf None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:42.316102 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887220ac-b56b-4e50-af60-3d0be47bebbf None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:42.318721 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887220ac-b56b-4e50-af60-3d0be47bebbf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:42.318721 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887220ac-b56b-4e50-af60-3d0be47bebbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:42.319827 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887220ac-b56b-4e50-af60-3d0be47bebbf None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:42.328401 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 103/196] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:47:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:42.329246 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887220ac-b56b-4e50-af60-3d0be47bebbf None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:42.375488 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:42.413134 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:42.420134 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b265c214-a144-4d51-831c-df318274956e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:42.433724 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887220ac-b56b-4e50-af60-3d0be47bebbf None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:42.454225 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4278530-2162-4be5-858e-9192dcbbbbe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:42.454829 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4278530-2162-4be5-858e-9192dcbbbbe6 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:42.455114 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4278530-2162-4be5-858e-9192dcbbbbe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:42.455388 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4278530-2162-4be5-858e-9192dcbbbbe6 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:42.456040 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4278530-2162-4be5-858e-9192dcbbbbe6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:42.457510 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4278530-2162-4be5-858e-9192dcbbbbe6 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:42.466198 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1426b-f973-4ef1-975f-293da580811a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:42.466824 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1426b-f973-4ef1-975f-293da580811a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:42.467100 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1426b-f973-4ef1-975f-293da580811a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:42.467368 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1426b-f973-4ef1-975f-293da580811a None admin] PATH_INFO: `/v3/users/5511ff4386ee4bb38c97add3954b95e0` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:42.468511 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1426b-f973-4ef1-975f-293da580811a None admin] RBAC: Authorizing `identity:get_user(user_id=5511ff4386ee4bb38c97add3954b95e0)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:42.470053 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1426b-f973-4ef1-975f-293da580811a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:42.471614 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 104/197] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:47:42 2022] GET /identity/v3/users/5511ff4386ee4bb38c97add3954b95e0 => generated 289 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:42.472370 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1426b-f973-4ef1-975f-293da580811a None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:42.543841 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 94/198] 10.222.0.27 () {60 vars in 1132 bytes} [Sun Sep 4 00:47:42 2022] POST /identity/v3/users => generated 297 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:47:42.545410 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4278530-2162-4be5-858e-9192dcbbbbe6 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:42.585675 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1426b-f973-4ef1-975f-293da580811a None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:42.599515 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:42.625308 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e11825-852f-43b0-8d62-398b56662f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:42.625928 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e11825-852f-43b0-8d62-398b56662f38 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:42.626328 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e11825-852f-43b0-8d62-398b56662f38 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:42.626624 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e11825-852f-43b0-8d62-398b56662f38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:42.667531 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4278530-2162-4be5-858e-9192dcbbbbe6 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:42.709287 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf26ec-06a9-4739-aee2-8d739ee13632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:42.710749 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf26ec-06a9-4739-aee2-8d739ee13632 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:42.711247 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf26ec-06a9-4739-aee2-8d739ee13632 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:42.711685 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf26ec-06a9-4739-aee2-8d739ee13632 None admin] PATH_INFO: `/v3/users/9c6861d39c0949349fb24061ef1c8249` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:42.712091 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e11825-852f-43b0-8d62-398b56662f38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:42.713230 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf26ec-06a9-4739-aee2-8d739ee13632 None admin] RBAC: Authorizing `identity:get_user(user_id=9c6861d39c0949349fb24061ef1c8249)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:42.713619 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e11825-852f-43b0-8d62-398b56662f38 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:42.714696 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf26ec-06a9-4739-aee2-8d739ee13632 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:42.716284 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 95/199] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:47:42 2022] GET /identity/v3/users/9c6861d39c0949349fb24061ef1c8249 => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:42.717133 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf26ec-06a9-4739-aee2-8d739ee13632 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:42.742538 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:42.850583 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 105/200] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:47:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:47:42.851638 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e11825-852f-43b0-8d62-398b56662f38 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:42.855273 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf26ec-06a9-4739-aee2-8d739ee13632 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:42.866715 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:42.892181 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e948ace-1893-4cf3-9e2c-5d8d101130c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:42.892823 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e948ace-1893-4cf3-9e2c-5d8d101130c2 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:42.893144 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e948ace-1893-4cf3-9e2c-5d8d101130c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:42.894107 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e948ace-1893-4cf3-9e2c-5d8d101130c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:42.968594 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e11825-852f-43b0-8d62-398b56662f38 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:43.009911 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f203ab8-9508-4395-af27-db2b6bd4bbab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:43.011433 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f203ab8-9508-4395-af27-db2b6bd4bbab None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:43.012061 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f203ab8-9508-4395-af27-db2b6bd4bbab None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:43.014783 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f203ab8-9508-4395-af27-db2b6bd4bbab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:43.019883 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e948ace-1893-4cf3-9e2c-5d8d101130c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:43.022856 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e948ace-1893-4cf3-9e2c-5d8d101130c2 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:43.046284 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:43.096275 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f203ab8-9508-4395-af27-db2b6bd4bbab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:43.098213 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f203ab8-9508-4395-af27-db2b6bd4bbab None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:43.111756 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:43.155717 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 96/201] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:47:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:47:43.161340 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e948ace-1893-4cf3-9e2c-5d8d101130c2 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:43.219627 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 106/202] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:47:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:47:43.220549 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f203ab8-9508-4395-af27-db2b6bd4bbab None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:43.308709 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e948ace-1893-4cf3-9e2c-5d8d101130c2 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:43.332449 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:43.341163 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f203ab8-9508-4395-af27-db2b6bd4bbab None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:43.352516 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:43.357287 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec40a5dd-b931-489a-a4c9-5ef3561ef0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:43.358512 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40a5dd-b931-489a-a4c9-5ef3561ef0bf None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:43.360980 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40a5dd-b931-489a-a4c9-5ef3561ef0bf None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:43.360980 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40a5dd-b931-489a-a4c9-5ef3561ef0bf None admin] PATH_INFO: `/v3/projects/5b12c253c6eb48c9a549a9a8436eced1` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:43.360980 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec40a5dd-b931-489a-a4c9-5ef3561ef0bf None admin] RBAC: Authorizing `identity:get_project(project_id=5b12c253c6eb48c9a549a9a8436eced1)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:43.362710 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec40a5dd-b931-489a-a4c9-5ef3561ef0bf None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:43.364968 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 97/203] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:47:43 2022] GET /identity/v3/projects/5b12c253c6eb48c9a549a9a8436eced1 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:43.366559 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec40a5dd-b931-489a-a4c9-5ef3561ef0bf None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:43.382742 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55951974-81dd-4355-b9a7-d74d6f917271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:43.388664 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55951974-81dd-4355-b9a7-d74d6f917271 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:43.388664 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55951974-81dd-4355-b9a7-d74d6f917271 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:43.388664 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55951974-81dd-4355-b9a7-d74d6f917271 None admin] PATH_INFO: `/v3/projects/c8fe5139a2a24ec780814c7f39d5b437` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:43.388664 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55951974-81dd-4355-b9a7-d74d6f917271 None admin] RBAC: Authorizing `identity:get_project(project_id=c8fe5139a2a24ec780814c7f39d5b437)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:43.388664 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55951974-81dd-4355-b9a7-d74d6f917271 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:43.389870 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 107/204] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:47:43 2022] GET /identity/v3/projects/c8fe5139a2a24ec780814c7f39d5b437 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:43.390837 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55951974-81dd-4355-b9a7-d74d6f917271 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:43.485145 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec40a5dd-b931-489a-a4c9-5ef3561ef0bf None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:43.521291 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55951974-81dd-4355-b9a7-d74d6f917271 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:43.531582 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd9f38-80e7-482e-9abe-325e095c8525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:43.532221 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd9f38-80e7-482e-9abe-325e095c8525 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:43.532525 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd9f38-80e7-482e-9abe-325e095c8525 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:43.532808 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd9f38-80e7-482e-9abe-325e095c8525 None admin] PATH_INFO: `/v3/projects/5b12c253c6eb48c9a549a9a8436eced1` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:43.534291 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dd9f38-80e7-482e-9abe-325e095c8525 None admin] RBAC: Authorizing `identity:get_project(project_id=5b12c253c6eb48c9a549a9a8436eced1)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:43.536942 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dd9f38-80e7-482e-9abe-325e095c8525 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:43.539052 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 98/205] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:47:43 2022] GET /identity/v3/projects/5b12c253c6eb48c9a549a9a8436eced1 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:43.540130 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd9f38-80e7-482e-9abe-325e095c8525 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:43.565761 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26695775-45b5-4b80-bd24-8f4781ac8363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:43.566386 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26695775-45b5-4b80-bd24-8f4781ac8363 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:43.566671 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26695775-45b5-4b80-bd24-8f4781ac8363 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:43.566949 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26695775-45b5-4b80-bd24-8f4781ac8363 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:43.568173 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26695775-45b5-4b80-bd24-8f4781ac8363 None admin] RBAC: Adding query filter params (role.id=c8a59d34ee2f4cefa6940cac778260c7, scope.project.id=5b12c253c6eb48c9a549a9a8436eced1, user.id=9c6861d39c0949349fb24061ef1c8249) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:43.568505 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26695775-45b5-4b80-bd24-8f4781ac8363 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:43.570575 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26695775-45b5-4b80-bd24-8f4781ac8363 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:43.577712 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 108/206] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:47:43 2022] GET /identity/v3/role_assignments?user.id=9c6861d39c0949349fb24061ef1c8249&scope.project.id=5b12c253c6eb48c9a549a9a8436eced1&role.id=c8a59d34ee2f4cefa6940cac778260c7 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:43.578576 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26695775-45b5-4b80-bd24-8f4781ac8363 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:43.647766 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:43.668409 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:43.746917 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26695775-45b5-4b80-bd24-8f4781ac8363 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:43.767720 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd9f38-80e7-482e-9abe-325e095c8525 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:43.785416 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60be3eb-741f-49e1-ba99-b2d103e529a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:43.786065 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60be3eb-741f-49e1-ba99-b2d103e529a7 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:43.786347 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60be3eb-741f-49e1-ba99-b2d103e529a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:43.786617 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60be3eb-741f-49e1-ba99-b2d103e529a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:43.787727 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60be3eb-741f-49e1-ba99-b2d103e529a7 None admin] RBAC: Adding query filter params (role.id=7df9ebc56a2a4214b2b64a5941812ef0, scope.project.id=5b12c253c6eb48c9a549a9a8436eced1, user.id=9c6861d39c0949349fb24061ef1c8249) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:43.787845 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60be3eb-741f-49e1-ba99-b2d103e529a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:43.789516 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60be3eb-741f-49e1-ba99-b2d103e529a7 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:43.795569 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 109/207] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:47:43 2022] GET /identity/v3/role_assignments?user.id=9c6861d39c0949349fb24061ef1c8249&scope.project.id=5b12c253c6eb48c9a549a9a8436eced1&role.id=7df9ebc56a2a4214b2b64a5941812ef0 => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:43.840890 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f4aa4-2362-4b98-8183-e6ed83706b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:43.841950 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f4aa4-2362-4b98-8183-e6ed83706b65 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:43.842436 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f4aa4-2362-4b98-8183-e6ed83706b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:43.842921 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f4aa4-2362-4b98-8183-e6ed83706b65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:43.843892 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1f4aa4-2362-4b98-8183-e6ed83706b65 None admin] RBAC: Adding query filter params (role.id=8f7a2ac695c6482cbf48819cee250c22, scope.project.id=c8fe5139a2a24ec780814c7f39d5b437, user.id=5511ff4386ee4bb38c97add3954b95e0) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:43.844439 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1f4aa4-2362-4b98-8183-e6ed83706b65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:43.846236 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1f4aa4-2362-4b98-8183-e6ed83706b65 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:43.854436 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 99/208] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:47:43 2022] GET /identity/v3/role_assignments?user.id=5511ff4386ee4bb38c97add3954b95e0&scope.project.id=c8fe5139a2a24ec780814c7f39d5b437&role.id=8f7a2ac695c6482cbf48819cee250c22 => generated 608 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:44.722784 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cfe9dc-f172-4396-8cd2-c58a6904a6de None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:44.723297 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cfe9dc-f172-4396-8cd2-c58a6904a6de None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:44.723446 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cfe9dc-f172-4396-8cd2-c58a6904a6de None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:44.724753 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 110/209] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:44.729966 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd24dc-09c8-462f-b0af-80e6e7c7463e None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:44.730386 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd24dc-09c8-462f-b0af-80e6e7c7463e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:44.730556 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd24dc-09c8-462f-b0af-80e6e7c7463e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:44.749688 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-cebd24dc-09c8-462f-b0af-80e6e7c7463e None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:44.750890 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-cebd24dc-09c8-462f-b0af-80e6e7c7463e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:44.800423 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c138c-b4b6-40b9-a06d-93264ed2833c None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:44.800744 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c138c-b4b6-40b9-a06d-93264ed2833c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:44.801008 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c138c-b4b6-40b9-a06d-93264ed2833c None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:44.802880 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 111/210] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:44.808417 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e48543-b119-4a82-b3ba-f26da98264d6 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:44.808732 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e48543-b119-4a82-b3ba-f26da98264d6 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:44.809001 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e48543-b119-4a82-b3ba-f26da98264d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:44.826305 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 100/211] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:44.829234 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-e9e48543-b119-4a82-b3ba-f26da98264d6 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:44.830466 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e9e48543-b119-4a82-b3ba-f26da98264d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:44.901790 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 112/212] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 00:47:45.107741 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd24dc-09c8-462f-b0af-80e6e7c7463e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:45.159681 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:45.180057 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e48543-b119-4a82-b3ba-f26da98264d6 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:45.216532 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd24dc-09c8-462f-b0af-80e6e7c7463e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:45.216980 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:45.254614 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078ca6b-2207-4b15-865f-ffbff6530f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:45.255525 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078ca6b-2207-4b15-865f-ffbff6530f0a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:45.255812 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078ca6b-2207-4b15-865f-ffbff6530f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:45.256151 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078ca6b-2207-4b15-865f-ffbff6530f0a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:45.257320 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8078ca6b-2207-4b15-865f-ffbff6530f0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:45.258844 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8078ca6b-2207-4b15-865f-ffbff6530f0a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:45.260433 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 101/213] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:47:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:45.266118 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078ca6b-2207-4b15-865f-ffbff6530f0a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:45.289159 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e48543-b119-4a82-b3ba-f26da98264d6 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:45.326345 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624cbc7e-7d29-4003-84ce-fb1b51642bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:45.326987 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624cbc7e-7d29-4003-84ce-fb1b51642bc8 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:45.327264 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624cbc7e-7d29-4003-84ce-fb1b51642bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:45.327535 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624cbc7e-7d29-4003-84ce-fb1b51642bc8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:45.339910 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624cbc7e-7d29-4003-84ce-fb1b51642bc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:47:45.346930 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-624cbc7e-7d29-4003-84ce-fb1b51642bc8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:47:45.346930 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:45.346930 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:45.346930 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:45.346930 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:45.346930 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:45.346930 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:45.346930 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:45.346930 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:45.346930 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:45.346930 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:45.346930 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:45.346930 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:47:45.346930 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:47:45.346930 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:47:45.346930 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:47:45.349635 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:47:45.349635 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:47:45.349635 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:47:45.349635 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:47:45.349635 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:45.349635 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:45.349635 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:45.349635 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:45.349635 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:45.349635 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:45.349635 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:45.349635 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:45.349635 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:45.349635 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:45.349635 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:45.349635 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:45.350976 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:45.350976 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:45.350976 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:45.350976 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:45.350976 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:47:45.350976 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:47:45.350976 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:47:45.350976 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:47:45.350976 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:47:45.350976 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:47:45.350976 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:47:45.350976 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:47:45.350976 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 113/214] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:47:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:47:45.360570 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624cbc7e-7d29-4003-84ce-fb1b51642bc8 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:45.382728 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078ca6b-2207-4b15-865f-ffbff6530f0a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:45.401968 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:45.423253 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96970c15-7413-4588-8043-62576dd4cd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:45.423981 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96970c15-7413-4588-8043-62576dd4cd30 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:45.424607 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96970c15-7413-4588-8043-62576dd4cd30 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:45.424959 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96970c15-7413-4588-8043-62576dd4cd30 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:45.426296 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96970c15-7413-4588-8043-62576dd4cd30 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:45.429542 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96970c15-7413-4588-8043-62576dd4cd30 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:45.457255 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:45.463853 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624cbc7e-7d29-4003-84ce-fb1b51642bc8 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:45.498345 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49c54de-4ffc-486a-8520-f33c69c53de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:45.499007 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c54de-4ffc-486a-8520-f33c69c53de6 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:45.499286 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c54de-4ffc-486a-8520-f33c69c53de6 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:45.499560 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c54de-4ffc-486a-8520-f33c69c53de6 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:45.500228 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49c54de-4ffc-486a-8520-f33c69c53de6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:45.500553 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49c54de-4ffc-486a-8520-f33c69c53de6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:45.502047 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49c54de-4ffc-486a-8520-f33c69c53de6 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:45.510703 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 114/215] 10.222.0.27 () {56 vars in 1101 bytes} [Sun Sep 4 00:47:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:45.516677 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49c54de-4ffc-486a-8520-f33c69c53de6 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:45.540739 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 102/216] 10.222.0.27 () {60 vars in 1132 bytes} [Sun Sep 4 00:47:45 2022] POST /identity/v3/users => generated 297 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 00:47:45.644221 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49c54de-4ffc-486a-8520-f33c69c53de6 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:45.673777 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d1ab2-e3ef-4a53-af1b-c497834299a8 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:45.674337 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d1ab2-e3ef-4a53-af1b-c497834299a8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:45.675779 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d1ab2-e3ef-4a53-af1b-c497834299a8 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:45.676792 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 103/217] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:45.682889 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed605b-1acc-423f-a5a9-67b821e32f5d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:45.683285 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed605b-1acc-423f-a5a9-67b821e32f5d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:45.683684 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed605b-1acc-423f-a5a9-67b821e32f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:45.694911 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d26621-e89d-461a-8cb5-e7005d599515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:45.695838 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d26621-e89d-461a-8cb5-e7005d599515 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:45.696292 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d26621-e89d-461a-8cb5-e7005d599515 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:45.696756 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d26621-e89d-461a-8cb5-e7005d599515 None admin] PATH_INFO: `/v3/users/e133dde4c9cb4debbefdd320b783c004` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:45.706978 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-69ed605b-1acc-423f-a5a9-67b821e32f5d None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:45.709590 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-69ed605b-1acc-423f-a5a9-67b821e32f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:45.716074 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d26621-e89d-461a-8cb5-e7005d599515 None admin] RBAC: Authorizing `identity:get_user(user_id=e133dde4c9cb4debbefdd320b783c004)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:45.717686 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d26621-e89d-461a-8cb5-e7005d599515 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:45.719549 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 115/218] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:47:45 2022] GET /identity/v3/users/e133dde4c9cb4debbefdd320b783c004 => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:45.725657 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d26621-e89d-461a-8cb5-e7005d599515 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:45.745169 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:45.808599 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 104/219] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 00:47:45.813978 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420df12-3e40-4d26-970c-2b1018f26b26 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:45.814525 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420df12-3e40-4d26-970c-2b1018f26b26 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:45.814995 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420df12-3e40-4d26-970c-2b1018f26b26 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:45.816990 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 105/220] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:45.822631 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ffc537-5462-4936-9379-247c88a12e0d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:45.822915 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ffc537-5462-4936-9379-247c88a12e0d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:45.823180 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ffc537-5462-4936-9379-247c88a12e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:45.844295 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-40ffc537-5462-4936-9379-247c88a12e0d None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:45.846554 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-40ffc537-5462-4936-9379-247c88a12e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:45.853102 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d26621-e89d-461a-8cb5-e7005d599515 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:45.899313 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af397a6d-bb80-4c57-8b07-5c2bbaab7207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:45.899863 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af397a6d-bb80-4c57-8b07-5c2bbaab7207 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:45.900140 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af397a6d-bb80-4c57-8b07-5c2bbaab7207 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:45.900409 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af397a6d-bb80-4c57-8b07-5c2bbaab7207 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:45.931556 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 106/221] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 00:47:46.033780 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af397a6d-bb80-4c57-8b07-5c2bbaab7207 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:46.035255 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af397a6d-bb80-4c57-8b07-5c2bbaab7207 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:46.055745 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:46.108822 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ffc537-5462-4936-9379-247c88a12e0d None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:46.148258 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 116/222] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:47:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 04 00:47:46.153794 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af397a6d-bb80-4c57-8b07-5c2bbaab7207 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:46.181317 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:46.295318 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ffc537-5462-4936-9379-247c88a12e0d None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:46.320916 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af397a6d-bb80-4c57-8b07-5c2bbaab7207 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:46.333662 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd58a83c-a1eb-4965-b1e6-e1c78c57c269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:46.334395 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58a83c-a1eb-4965-b1e6-e1c78c57c269 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:46.334786 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58a83c-a1eb-4965-b1e6-e1c78c57c269 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:46.335066 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58a83c-a1eb-4965-b1e6-e1c78c57c269 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:46.337335 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:46.347643 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd58a83c-a1eb-4965-b1e6-e1c78c57c269 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:47:46.352960 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-cd58a83c-a1eb-4965-b1e6-e1c78c57c269 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:47:46.352960 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:46.352960 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:46.352960 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:46.352960 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:46.352960 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:46.352960 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:46.352960 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:46.352960 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:46.352960 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:46.352960 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:46.352960 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:46.352960 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:47:46.352960 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:47:46.352960 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:47:46.352960 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:47:46.354905 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:47:46.354905 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:47:46.354905 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:47:46.354905 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:47:46.354905 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:46.354905 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:46.354905 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:46.354905 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:46.354905 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:46.354905 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:46.354905 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:46.354905 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:46.354905 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:46.354905 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:46.354905 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:46.354905 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:46.355711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:46.355711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:46.355711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:46.355711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:46.355711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:47:46.355711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:47:46.355711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:47:46.355711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:47:46.355711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:47:46.355711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:47:46.355711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:47:46.355711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:47:46.355711 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 107/223] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:47:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:47:46.355711 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd58a83c-a1eb-4965-b1e6-e1c78c57c269 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:46.356663 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a3572-54ee-4a1e-bdae-678942b2fd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:46.356663 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a3572-54ee-4a1e-bdae-678942b2fd05 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:46.356663 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a3572-54ee-4a1e-bdae-678942b2fd05 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:46.356883 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a3572-54ee-4a1e-bdae-678942b2fd05 None admin] PATH_INFO: `/v3/projects/5b12c253c6eb48c9a549a9a8436eced1` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:46.359722 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4a3572-54ee-4a1e-bdae-678942b2fd05 None admin] RBAC: Authorizing `identity:get_project(project_id=5b12c253c6eb48c9a549a9a8436eced1)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:46.359722 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4a3572-54ee-4a1e-bdae-678942b2fd05 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:46.361410 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 117/224] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:47:46 2022] GET /identity/v3/projects/5b12c253c6eb48c9a549a9a8436eced1 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:47:46.362260 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a3572-54ee-4a1e-bdae-678942b2fd05 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:46.465190 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a3572-54ee-4a1e-bdae-678942b2fd05 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:46.480137 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd58a83c-a1eb-4965-b1e6-e1c78c57c269 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:46.499833 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435d4f7-139f-4cda-91b6-15caed0cb118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:46.500519 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435d4f7-139f-4cda-91b6-15caed0cb118 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:46.500807 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435d4f7-139f-4cda-91b6-15caed0cb118 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:46.501080 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435d4f7-139f-4cda-91b6-15caed0cb118 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:46.501748 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7435d4f7-139f-4cda-91b6-15caed0cb118 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:46.502049 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7435d4f7-139f-4cda-91b6-15caed0cb118 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:46.503426 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7435d4f7-139f-4cda-91b6-15caed0cb118 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:46.512272 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 118/225] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:47:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:46.513108 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435d4f7-139f-4cda-91b6-15caed0cb118 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:46.516610 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:46.527225 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ef19ad-ed59-47bd-a0f9-3e8a778061f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:46.528703 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef19ad-ed59-47bd-a0f9-3e8a778061f5 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:46.531611 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef19ad-ed59-47bd-a0f9-3e8a778061f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:46.531611 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef19ad-ed59-47bd-a0f9-3e8a778061f5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:46.541434 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ef19ad-ed59-47bd-a0f9-3e8a778061f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:47:46.547261 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-49ef19ad-ed59-47bd-a0f9-3e8a778061f5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 00:47:46.547261 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:46.547261 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:46.547261 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:46.547261 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:46.547261 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:46.547261 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:46.547261 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:46.547261 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:46.547261 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:46.547261 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:46.547261 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:46.547261 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:47:46.547261 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:47:46.547261 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:47:46.547261 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:47:46.548406 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:47:46.548406 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:47:46.548406 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:47:46.548406 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:47:46.548406 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:46.548406 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:46.548406 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:46.548406 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:46.548406 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:46.548406 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:46.548406 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:46.548406 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:46.548406 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:46.548406 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:46.548406 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:46.548406 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:46.549223 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:46.549223 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:46.549223 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:46.549223 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:46.549223 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:47:46.549223 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:47:46.549223 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:47:46.549223 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:47:46.549223 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:47:46.549223 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:47:46.549223 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 00:47:46.549223 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:47:46.552899 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 108/226] 10.222.0.27 () {56 vars in 1114 bytes} [Sun Sep 4 00:47:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:47:46.554038 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ef19ad-ed59-47bd-a0f9-3e8a778061f5 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:46.561171 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:46.616366 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435d4f7-139f-4cda-91b6-15caed0cb118 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:46.650198 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c53d8-f018-4675-bbcf-2385c1351437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:46.650805 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c53d8-f018-4675-bbcf-2385c1351437 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:46.651085 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c53d8-f018-4675-bbcf-2385c1351437 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:46.651366 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c53d8-f018-4675-bbcf-2385c1351437 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:46.652091 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c53d8-f018-4675-bbcf-2385c1351437 None admin] RBAC: Adding query filter params (role.id=d1261c99e1d64f9da63adfff739561e8, scope.project.id=5b12c253c6eb48c9a549a9a8436eced1, user.id=e133dde4c9cb4debbefdd320b783c004) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:46.652398 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c53d8-f018-4675-bbcf-2385c1351437 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:46.653864 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c53d8-f018-4675-bbcf-2385c1351437 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:46.659285 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 119/227] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:47:46 2022] GET /identity/v3/role_assignments?user.id=e133dde4c9cb4debbefdd320b783c004&scope.project.id=5b12c253c6eb48c9a549a9a8436eced1&role.id=d1261c99e1d64f9da63adfff739561e8 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:46.664518 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c53d8-f018-4675-bbcf-2385c1351437 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:46.682336 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ef19ad-ed59-47bd-a0f9-3e8a778061f5 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:46.767415 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c7682-1de6-485a-9a86-6453ec54c1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:46.768060 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c7682-1de6-485a-9a86-6453ec54c1b6 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:46.768337 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c7682-1de6-485a-9a86-6453ec54c1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:46.768634 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c7682-1de6-485a-9a86-6453ec54c1b6 None admin] PATH_INFO: `/v3/users/9c6861d39c0949349fb24061ef1c8249` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:46.769771 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c7682-1de6-485a-9a86-6453ec54c1b6 None admin] RBAC: Authorizing `identity:get_user(user_id=9c6861d39c0949349fb24061ef1c8249)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:46.771284 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c7682-1de6-485a-9a86-6453ec54c1b6 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:46.772948 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 109/228] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:47:46 2022] GET /identity/v3/users/9c6861d39c0949349fb24061ef1c8249 => generated 283 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:46.778520 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c7682-1de6-485a-9a86-6453ec54c1b6 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:46.816837 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:46.835411 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c53d8-f018-4675-bbcf-2385c1351437 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:46.852776 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:46.911890 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d151ac-81c0-48d0-8fd3-c7008139496f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:46.912538 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d151ac-81c0-48d0-8fd3-c7008139496f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:46.912823 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d151ac-81c0-48d0-8fd3-c7008139496f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:46.913101 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d151ac-81c0-48d0-8fd3-c7008139496f None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:46.913866 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d151ac-81c0-48d0-8fd3-c7008139496f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:46.914176 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d151ac-81c0-48d0-8fd3-c7008139496f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:46.915643 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d151ac-81c0-48d0-8fd3-c7008139496f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:46.922141 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c7682-1de6-485a-9a86-6453ec54c1b6 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:46.934909 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 120/229] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:47:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:46.940195 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d151ac-81c0-48d0-8fd3-c7008139496f None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:46.961431 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31840c43-dc9a-41d5-ab25-6f3169df1b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:46.962188 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31840c43-dc9a-41d5-ab25-6f3169df1b65 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:46.962484 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31840c43-dc9a-41d5-ab25-6f3169df1b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:46.962765 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31840c43-dc9a-41d5-ab25-6f3169df1b65 None admin] PATH_INFO: `/v3/projects/5b12c253c6eb48c9a549a9a8436eced1` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:46.964040 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31840c43-dc9a-41d5-ab25-6f3169df1b65 None admin] RBAC: Authorizing `identity:get_project(project_id=5b12c253c6eb48c9a549a9a8436eced1)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:46.965573 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31840c43-dc9a-41d5-ab25-6f3169df1b65 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:46.967554 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 110/230] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:47:46 2022] GET /identity/v3/projects/5b12c253c6eb48c9a549a9a8436eced1 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:47:46.974201 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31840c43-dc9a-41d5-ab25-6f3169df1b65 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:47.096148 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:47.113139 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31840c43-dc9a-41d5-ab25-6f3169df1b65 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:47.150811 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a792839-ac60-49bc-a26a-35adc43ebeb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:47.151475 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a792839-ac60-49bc-a26a-35adc43ebeb5 None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:47.151755 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a792839-ac60-49bc-a26a-35adc43ebeb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:47.152030 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a792839-ac60-49bc-a26a-35adc43ebeb5 None admin] PATH_INFO: `/v3/projects/5b12c253c6eb48c9a549a9a8436eced1/users/9c6861d39c0949349fb24061ef1c8249/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:47.155748 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a792839-ac60-49bc-a26a-35adc43ebeb5 None admin] RBAC: Authorizing `identity:create_grant(project_id=5b12c253c6eb48c9a549a9a8436eced1, user_id=9c6861d39c0949349fb24061ef1c8249, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:47.157204 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a792839-ac60-49bc-a26a-35adc43ebeb5 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:47.162774 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d151ac-81c0-48d0-8fd3-c7008139496f None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:47.200963 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2311085-5358-4885-902e-80a6484a3db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:47.201605 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2311085-5358-4885-902e-80a6484a3db5 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:47.201934 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2311085-5358-4885-902e-80a6484a3db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:47.202210 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2311085-5358-4885-902e-80a6484a3db5 None admin] PATH_INFO: `/v3/users/9c6861d39c0949349fb24061ef1c8249` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:47.203422 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2311085-5358-4885-902e-80a6484a3db5 None admin] RBAC: Authorizing `identity:get_user(user_id=9c6861d39c0949349fb24061ef1c8249)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:47.204928 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2311085-5358-4885-902e-80a6484a3db5 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:47.206935 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 121/231] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:47:46 2022] GET /identity/v3/users/9c6861d39c0949349fb24061ef1c8249 => generated 283 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:47.212738 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2311085-5358-4885-902e-80a6484a3db5 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:47.259428 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 111/232] 10.222.0.27 () {58 vars in 1439 bytes} [Sun Sep 4 00:47:46 2022] PUT /identity/v3/projects/5b12c253c6eb48c9a549a9a8436eced1/users/9c6861d39c0949349fb24061ef1c8249/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 00:47:47.273282 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:47.347839 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2311085-5358-4885-902e-80a6484a3db5 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:47.410565 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5995bd07-e457-4516-b8ef-7e7ba4dd4862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:47.411746 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995bd07-e457-4516-b8ef-7e7ba4dd4862 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:47.412025 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995bd07-e457-4516-b8ef-7e7ba4dd4862 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:47.412297 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995bd07-e457-4516-b8ef-7e7ba4dd4862 None admin] PATH_INFO: `/v3/projects/5b12c253c6eb48c9a549a9a8436eced1` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:47.413480 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5995bd07-e457-4516-b8ef-7e7ba4dd4862 None admin] RBAC: Authorizing `identity:get_project(project_id=5b12c253c6eb48c9a549a9a8436eced1)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:47.415020 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5995bd07-e457-4516-b8ef-7e7ba4dd4862 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:47.416718 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 122/233] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:47:47 2022] GET /identity/v3/projects/5b12c253c6eb48c9a549a9a8436eced1 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:47.423020 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a792839-ac60-49bc-a26a-35adc43ebeb5 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:47.544193 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a792839-ac60-49bc-a26a-35adc43ebeb5 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:47.579707 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95115b6e-6733-468a-a713-ebe072cd7e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:47.580368 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95115b6e-6733-468a-a713-ebe072cd7e27 None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:47.580746 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95115b6e-6733-468a-a713-ebe072cd7e27 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:47.581025 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95115b6e-6733-468a-a713-ebe072cd7e27 None admin] PATH_INFO: `/v3/projects/5b12c253c6eb48c9a549a9a8436eced1/users/9c6861d39c0949349fb24061ef1c8249/roles/7df9ebc56a2a4214b2b64a5941812ef0` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:47.583294 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95115b6e-6733-468a-a713-ebe072cd7e27 None admin] RBAC: Authorizing `identity:create_grant(project_id=5b12c253c6eb48c9a549a9a8436eced1, user_id=9c6861d39c0949349fb24061ef1c8249, role_id=7df9ebc56a2a4214b2b64a5941812ef0)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:47.584945 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95115b6e-6733-468a-a713-ebe072cd7e27 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:47.698811 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 112/234] 10.222.0.27 () {58 vars in 1439 bytes} [Sun Sep 4 00:47:47 2022] PUT /identity/v3/projects/5b12c253c6eb48c9a549a9a8436eced1/users/9c6861d39c0949349fb24061ef1c8249/roles/7df9ebc56a2a4214b2b64a5941812ef0 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 00:47:48.006859 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e78c3-d179-4710-bb0e-2662dc0c289a None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:48.007366 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e78c3-d179-4710-bb0e-2662dc0c289a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:48.007468 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e78c3-d179-4710-bb0e-2662dc0c289a None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:48.009085 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 123/235] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:48.014695 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869bc647-2bcd-403a-b6b6-4e1c3498883a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:48.015156 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869bc647-2bcd-403a-b6b6-4e1c3498883a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:48.015233 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869bc647-2bcd-403a-b6b6-4e1c3498883a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:48.036933 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-869bc647-2bcd-403a-b6b6-4e1c3498883a None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:48.038452 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-869bc647-2bcd-403a-b6b6-4e1c3498883a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:48.040886 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:48.128820 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 113/236] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:48.233810 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291fc01-6a6e-4dc2-8eb5-de52d89f3322 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:48.234347 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291fc01-6a6e-4dc2-8eb5-de52d89f3322 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:48.234818 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291fc01-6a6e-4dc2-8eb5-de52d89f3322 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:48.236638 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 124/237] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:48.242341 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edb8e1-d3da-4fe7-9570-cb1acef3b7af None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:48.242635 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edb8e1-d3da-4fe7-9570-cb1acef3b7af None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:48.242921 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edb8e1-d3da-4fe7-9570-cb1acef3b7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:48.262704 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-49edb8e1-d3da-4fe7-9570-cb1acef3b7af None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:48.263825 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-49edb8e1-d3da-4fe7-9570-cb1acef3b7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:48.358963 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 114/238] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:48.505813 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291fc01-6a6e-4dc2-8eb5-de52d89f3322 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:48.639807 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49edb8e1-d3da-4fe7-9570-cb1acef3b7af None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:48.650100 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291fc01-6a6e-4dc2-8eb5-de52d89f3322 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:48.663507 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:48.709295 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0728484f-970d-4c53-a50c-3847a0a439c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:48.710307 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728484f-970d-4c53-a50c-3847a0a439c5 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:48.710808 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728484f-970d-4c53-a50c-3847a0a439c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:48.711298 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728484f-970d-4c53-a50c-3847a0a439c5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:48.724773 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:48.733368 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0728484f-970d-4c53-a50c-3847a0a439c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:47:48.738437 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-0728484f-970d-4c53-a50c-3847a0a439c5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 00:47:48.738437 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:48.738437 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:48.738437 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:48.738437 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:48.738437 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:48.738437 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:48.738437 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:48.738437 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:48.738437 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:48.738437 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:48.738437 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:48.738437 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:47:48.738437 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:47:48.738437 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:47:48.738437 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:47:48.740789 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:47:48.740789 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:47:48.740789 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:47:48.740789 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:47:48.740789 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:48.740789 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:48.740789 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:48.740789 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:48.740789 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:48.740789 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:48.740789 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:48.740789 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:48.740789 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:48.740789 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:48.740789 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:48.740789 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:48.741555 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:48.741555 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:48.741555 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:48.741555 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:48.741555 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:47:48.741555 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:47:48.741555 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:47:48.741555 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:47:48.741555 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:47:48.741555 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:47:48.741555 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 00:47:48.741555 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:47:48.742444 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 125/239] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:47:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:47:48.749680 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0728484f-970d-4c53-a50c-3847a0a439c5 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:48.773661 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49edb8e1-d3da-4fe7-9570-cb1acef3b7af None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:48.818913 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceedc018-08d1-42f3-8a14-e0f15ce70794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:48.819848 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceedc018-08d1-42f3-8a14-e0f15ce70794 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:48.820413 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceedc018-08d1-42f3-8a14-e0f15ce70794 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:48.820907 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceedc018-08d1-42f3-8a14-e0f15ce70794 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:48.822182 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceedc018-08d1-42f3-8a14-e0f15ce70794 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:48.824079 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceedc018-08d1-42f3-8a14-e0f15ce70794 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:48.825867 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 115/240] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:47:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:47:48.827477 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc5e6b-48c4-4e3d-b214-b973dac65a5e None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:48.827956 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc5e6b-48c4-4e3d-b214-b973dac65a5e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:48.828425 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc5e6b-48c4-4e3d-b214-b973dac65a5e None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:48.830220 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 116/241] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:48.831898 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc5e6b-48c4-4e3d-b214-b973dac65a5e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:48.898122 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0728484f-970d-4c53-a50c-3847a0a439c5 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:48.940630 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d7fb7-8632-4d0e-9f16-8b3dbbb26077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:48.941227 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d7fb7-8632-4d0e-9f16-8b3dbbb26077 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:48.941641 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d7fb7-8632-4d0e-9f16-8b3dbbb26077 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:48.941980 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d7fb7-8632-4d0e-9f16-8b3dbbb26077 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:48.942741 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040d7fb7-8632-4d0e-9f16-8b3dbbb26077 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:48.943045 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040d7fb7-8632-4d0e-9f16-8b3dbbb26077 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:48.944896 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040d7fb7-8632-4d0e-9f16-8b3dbbb26077 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:48.948778 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc5e6b-48c4-4e3d-b214-b973dac65a5e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:48.954640 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 126/242] 10.222.0.27 () {56 vars in 1101 bytes} [Sun Sep 4 00:47:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:47:48.955815 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4f76f-4212-486b-b3fc-ed0e46c99205 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:48.956099 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4f76f-4212-486b-b3fc-ed0e46c99205 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:48.956375 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4f76f-4212-486b-b3fc-ed0e46c99205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:48.978245 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-8ba4f76f-4212-486b-b3fc-ed0e46c99205 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:48.979591 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8ba4f76f-4212-486b-b3fc-ed0e46c99205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:48.988438 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a68a0f-8324-4011-a3ed-eec85df19873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:48.990159 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a68a0f-8324-4011-a3ed-eec85df19873 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:48.990640 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a68a0f-8324-4011-a3ed-eec85df19873 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:48.991214 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a68a0f-8324-4011-a3ed-eec85df19873 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:48.992107 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a68a0f-8324-4011-a3ed-eec85df19873 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:48.993971 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a68a0f-8324-4011-a3ed-eec85df19873 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:49.003844 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:49.006741 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:49.050682 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 127/243] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:49.051953 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4f76f-4212-486b-b3fc-ed0e46c99205 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:49.085440 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 117/244] 10.222.0.27 () {60 vars in 1132 bytes} [Sun Sep 4 00:47:48 2022] POST /identity/v3/users => generated 293 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:47:49.171689 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4f76f-4212-486b-b3fc-ed0e46c99205 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:49.210460 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f922cb-ccc9-401e-afe8-093d47f29552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:49.211477 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f922cb-ccc9-401e-afe8-093d47f29552 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:49.211979 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f922cb-ccc9-401e-afe8-093d47f29552 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:49.212481 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f922cb-ccc9-401e-afe8-093d47f29552 None admin] PATH_INFO: `/v3/users/d11f31e750454372b47b35c41fcc9726` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:49.228152 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f922cb-ccc9-401e-afe8-093d47f29552 None admin] RBAC: Authorizing `identity:get_user(user_id=d11f31e750454372b47b35c41fcc9726)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:49.230127 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f922cb-ccc9-401e-afe8-093d47f29552 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:49.232986 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 128/245] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:47:49 2022] GET /identity/v3/users/d11f31e750454372b47b35c41fcc9726 => generated 297 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:49.241197 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a68a0f-8324-4011-a3ed-eec85df19873 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:49.326889 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f922cb-ccc9-401e-afe8-093d47f29552 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:49.349990 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:49.414195 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a68a0f-8324-4011-a3ed-eec85df19873 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:49.461689 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb41a4-4dab-424e-a01c-35ba184033db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:49.462860 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb41a4-4dab-424e-a01c-35ba184033db None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:49.463705 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb41a4-4dab-424e-a01c-35ba184033db None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:49.464413 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb41a4-4dab-424e-a01c-35ba184033db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:49.466310 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f922cb-ccc9-401e-afe8-093d47f29552 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:49.514307 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d52dd-b244-4623-8998-450b0b7facc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:49.517765 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d52dd-b244-4623-8998-450b0b7facc1 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:49.518050 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d52dd-b244-4623-8998-450b0b7facc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:49.518668 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d52dd-b244-4623-8998-450b0b7facc1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:49.530628 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d52dd-b244-4623-8998-450b0b7facc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:47:49.536524 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-9f6d52dd-b244-4623-8998-450b0b7facc1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:47:49.536524 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:49.536524 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:49.536524 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:49.536524 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:49.536524 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:49.536524 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:49.536524 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:49.536524 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:49.536524 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:49.536524 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:49.536524 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:49.536524 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:47:49.536524 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:47:49.536524 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:47:49.536524 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:47:49.537422 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:47:49.537422 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:47:49.537422 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:47:49.537422 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:47:49.537422 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:49.537422 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:49.537422 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:49.537422 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:49.537422 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:49.537422 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:49.537422 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:49.537422 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:49.537422 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:49.537422 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:49.537422 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:49.537422 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:49.538180 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:49.538180 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:49.538180 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:49.538180 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:49.538180 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:47:49.538180 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:47:49.538180 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:47:49.538180 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:47:49.538180 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:47:49.538180 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:47:49.538180 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:47:49.538180 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:47:49.538180 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 129/246] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:47:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:47:49.545664 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d52dd-b244-4623-8998-450b0b7facc1 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:49.549145 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:49.564478 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecb41a4-4dab-424e-a01c-35ba184033db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:49.566476 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecb41a4-4dab-424e-a01c-35ba184033db None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:49.686606 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:49.697841 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 118/247] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:47:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:47:49.703173 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb41a4-4dab-424e-a01c-35ba184033db None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:49.722319 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d52dd-b244-4623-8998-450b0b7facc1 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:49.766883 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b908efd0-b42d-41c3-8f32-107dd8b4c736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:49.767507 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908efd0-b42d-41c3-8f32-107dd8b4c736 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:49.767791 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908efd0-b42d-41c3-8f32-107dd8b4c736 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:49.768065 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908efd0-b42d-41c3-8f32-107dd8b4c736 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:49.768810 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b908efd0-b42d-41c3-8f32-107dd8b4c736 None admin] RBAC: Adding query filter params (name=member) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:49.769232 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b908efd0-b42d-41c3-8f32-107dd8b4c736 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:49.771332 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b908efd0-b42d-41c3-8f32-107dd8b4c736 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:49.780746 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 130/248] 10.222.0.27 () {56 vars in 1101 bytes} [Sun Sep 4 00:47:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:49.782132 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38643e67-fb72-436a-8f98-4d6bb66192c7 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:49.782419 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38643e67-fb72-436a-8f98-4d6bb66192c7 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:49.782710 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38643e67-fb72-436a-8f98-4d6bb66192c7 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:49.784279 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 131/249] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:49.789638 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b353b7-8b89-400c-9fb1-6d1487b471a1 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:49.790301 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b353b7-8b89-400c-9fb1-6d1487b471a1 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:49.792174 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b353b7-8b89-400c-9fb1-6d1487b471a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:49.816706 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-29b353b7-8b89-400c-9fb1-6d1487b471a1 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:49.818834 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-29b353b7-8b89-400c-9fb1-6d1487b471a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:49.834328 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:49.844368 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb41a4-4dab-424e-a01c-35ba184033db None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:49.883873 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a777c0e7-9973-40e4-bfb5-0db20d9117a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:49.884537 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777c0e7-9973-40e4-bfb5-0db20d9117a0 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:49.884823 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777c0e7-9973-40e4-bfb5-0db20d9117a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:49.885099 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777c0e7-9973-40e4-bfb5-0db20d9117a0 None admin] PATH_INFO: `/v3/projects/5b12c253c6eb48c9a549a9a8436eced1` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:49.886468 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a777c0e7-9973-40e4-bfb5-0db20d9117a0 None admin] RBAC: Authorizing `identity:get_project(project_id=5b12c253c6eb48c9a549a9a8436eced1)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:49.887981 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a777c0e7-9973-40e4-bfb5-0db20d9117a0 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:49.890308 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 119/250] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:47:49 2022] GET /identity/v3/projects/5b12c253c6eb48c9a549a9a8436eced1 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:49.890866 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a777c0e7-9973-40e4-bfb5-0db20d9117a0 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:49.893707 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 132/251] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 00:47:49.901519 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b353b7-8b89-400c-9fb1-6d1487b471a1 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:50.003929 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:50.005464 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a777c0e7-9973-40e4-bfb5-0db20d9117a0 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:50.044807 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a306114-458e-4067-b028-677766f29c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:50.045536 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a306114-458e-4067-b028-677766f29c9a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:50.045922 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a306114-458e-4067-b028-677766f29c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:50.046200 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a306114-458e-4067-b028-677766f29c9a None admin] PATH_INFO: `/v3/users/e133dde4c9cb4debbefdd320b783c004` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:50.047377 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a306114-458e-4067-b028-677766f29c9a None admin] RBAC: Authorizing `identity:get_user(user_id=e133dde4c9cb4debbefdd320b783c004)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:50.049060 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a306114-458e-4067-b028-677766f29c9a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:50.050957 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 120/252] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:47:49 2022] GET /identity/v3/users/e133dde4c9cb4debbefdd320b783c004 => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:50.056346 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b353b7-8b89-400c-9fb1-6d1487b471a1 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:50.057544 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a306114-458e-4067-b028-677766f29c9a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:50.093180 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f923a45-dc71-47de-bb8a-3e383c4881b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:50.093798 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f923a45-dc71-47de-bb8a-3e383c4881b8 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:50.094078 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f923a45-dc71-47de-bb8a-3e383c4881b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:50.094349 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f923a45-dc71-47de-bb8a-3e383c4881b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:50.095086 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f923a45-dc71-47de-bb8a-3e383c4881b8 None admin] RBAC: Adding query filter params (role.id=8f7a2ac695c6482cbf48819cee250c22, scope.project.id=5b12c253c6eb48c9a549a9a8436eced1, user.id=d11f31e750454372b47b35c41fcc9726) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:50.095389 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f923a45-dc71-47de-bb8a-3e383c4881b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:50.096889 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f923a45-dc71-47de-bb8a-3e383c4881b8 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:50.103413 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 133/253] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:47:49 2022] GET /identity/v3/role_assignments?user.id=d11f31e750454372b47b35c41fcc9726&scope.project.id=5b12c253c6eb48c9a549a9a8436eced1&role.id=8f7a2ac695c6482cbf48819cee250c22 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:50.114412 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:50.202920 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a306114-458e-4067-b028-677766f29c9a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:50.261418 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffcce7-a6ed-48c8-a128-8e53c6143a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:50.261955 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffcce7-a6ed-48c8-a128-8e53c6143a98 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:50.262230 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffcce7-a6ed-48c8-a128-8e53c6143a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:50.262501 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffcce7-a6ed-48c8-a128-8e53c6143a98 None admin] PATH_INFO: `/v3/projects/5b12c253c6eb48c9a549a9a8436eced1` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:50.263612 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeffcce7-a6ed-48c8-a128-8e53c6143a98 None admin] RBAC: Authorizing `identity:get_project(project_id=5b12c253c6eb48c9a549a9a8436eced1)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:50.265100 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeffcce7-a6ed-48c8-a128-8e53c6143a98 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:50.267027 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 121/254] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:47:50 2022] GET /identity/v3/projects/5b12c253c6eb48c9a549a9a8436eced1 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:50.273445 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f923a45-dc71-47de-bb8a-3e383c4881b8 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:50.381130 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f923a45-dc71-47de-bb8a-3e383c4881b8 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:50.419598 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f186f-dbf3-4fb2-a863-06ea8d632d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:50.420258 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f186f-dbf3-4fb2-a863-06ea8d632d03 None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:50.420590 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f186f-dbf3-4fb2-a863-06ea8d632d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:50.420866 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f186f-dbf3-4fb2-a863-06ea8d632d03 None admin] PATH_INFO: `/v3/projects/5b12c253c6eb48c9a549a9a8436eced1/users/e133dde4c9cb4debbefdd320b783c004/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:50.422794 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670f186f-dbf3-4fb2-a863-06ea8d632d03 None admin] RBAC: Authorizing `identity:create_grant(project_id=5b12c253c6eb48c9a549a9a8436eced1, user_id=e133dde4c9cb4debbefdd320b783c004, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:50.424258 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670f186f-dbf3-4fb2-a863-06ea8d632d03 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:50.427260 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:50.432245 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffcce7-a6ed-48c8-a128-8e53c6143a98 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:50.527595 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 134/255] 10.222.0.27 () {58 vars in 1439 bytes} [Sun Sep 4 00:47:50 2022] PUT /identity/v3/projects/5b12c253c6eb48c9a549a9a8436eced1/users/e133dde4c9cb4debbefdd320b783c004/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 00:47:50.561095 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:50.566175 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffcce7-a6ed-48c8-a128-8e53c6143a98 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:50.617020 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e51d301-48c1-40fb-bb77-b8fa8f1ca666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:50.618027 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51d301-48c1-40fb-bb77-b8fa8f1ca666 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:50.618505 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51d301-48c1-40fb-bb77-b8fa8f1ca666 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:50.619006 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51d301-48c1-40fb-bb77-b8fa8f1ca666 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:50.639995 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e51d301-48c1-40fb-bb77-b8fa8f1ca666 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:47:50.646192 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-0e51d301-48c1-40fb-bb77-b8fa8f1ca666 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:47:50.646192 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:50.646192 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:50.646192 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:50.646192 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:50.646192 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:50.646192 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:50.646192 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:50.646192 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:50.646192 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:50.646192 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:50.646192 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:50.646192 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:47:50.646192 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:47:50.646192 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:47:50.646192 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:47:50.647119 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:47:50.647119 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:47:50.647119 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:47:50.647119 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:47:50.647119 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:50.647119 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:50.647119 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:50.647119 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:50.647119 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:50.647119 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:50.647119 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:50.647119 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:50.647119 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:50.647119 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:50.647119 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:50.647119 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:50.647879 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:50.647879 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:50.647879 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:50.647879 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:50.647879 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:47:50.647879 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:47:50.647879 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:47:50.647879 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:47:50.647879 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:47:50.647879 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:47:50.647879 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:47:50.647879 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:47:50.650724 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 122/256] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:47:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:47:50.656840 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f186f-dbf3-4fb2-a863-06ea8d632d03 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:50.735613 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae280b8-eb02-41b1-b07b-601a92391e20 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:50.735922 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae280b8-eb02-41b1-b07b-601a92391e20 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:50.736183 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae280b8-eb02-41b1-b07b-601a92391e20 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:50.737694 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 123/257] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:50.743867 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184ac2a-99c1-4c9a-a962-7c4a2f86d82e None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:50.744144 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184ac2a-99c1-4c9a-a962-7c4a2f86d82e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:50.744422 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184ac2a-99c1-4c9a-a962-7c4a2f86d82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:50.789572 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f186f-dbf3-4fb2-a863-06ea8d632d03 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:50.796487 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-d184ac2a-99c1-4c9a-a962-7c4a2f86d82e None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:50.796487 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d184ac2a-99c1-4c9a-a962-7c4a2f86d82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:50.851369 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc12c80-44c7-4a94-b86f-922f9ec43016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:50.852340 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc12c80-44c7-4a94-b86f-922f9ec43016 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:50.852861 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc12c80-44c7-4a94-b86f-922f9ec43016 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:50.853450 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc12c80-44c7-4a94-b86f-922f9ec43016 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:50.854330 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc12c80-44c7-4a94-b86f-922f9ec43016 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:50.854838 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc12c80-44c7-4a94-b86f-922f9ec43016 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:50.856418 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc12c80-44c7-4a94-b86f-922f9ec43016 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:50.870616 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 135/258] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:47:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:50.876197 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc12c80-44c7-4a94-b86f-922f9ec43016 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:50.944073 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 124/259] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:50.974234 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:51.033471 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc12c80-44c7-4a94-b86f-922f9ec43016 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:51.081235 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f3310-091e-4812-b918-6bfac095aca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:51.084303 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f3310-091e-4812-b918-6bfac095aca4 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:51.084856 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f3310-091e-4812-b918-6bfac095aca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:51.085330 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f3310-091e-4812-b918-6bfac095aca4 None admin] PATH_INFO: `/v3/users/9c6861d39c0949349fb24061ef1c8249` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:51.086738 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f3310-091e-4812-b918-6bfac095aca4 None admin] RBAC: Authorizing `identity:get_user(user_id=9c6861d39c0949349fb24061ef1c8249)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:51.088426 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f3310-091e-4812-b918-6bfac095aca4 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:51.090616 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 136/260] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:47:50 2022] GET /identity/v3/users/9c6861d39c0949349fb24061ef1c8249 => generated 283 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:51.096107 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d184ac2a-99c1-4c9a-a962-7c4a2f86d82e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:51.243879 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:51.260951 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d184ac2a-99c1-4c9a-a962-7c4a2f86d82e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:51.297492 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b6f65-7343-4b52-be17-b1658e6393e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:51.298410 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b6f65-7343-4b52-be17-b1658e6393e6 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:51.298863 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b6f65-7343-4b52-be17-b1658e6393e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:51.299282 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b6f65-7343-4b52-be17-b1658e6393e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:51.370376 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b6f65-7343-4b52-be17-b1658e6393e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:51.372058 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b6f65-7343-4b52-be17-b1658e6393e6 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:51.461671 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:51.477850 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 125/261] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:47:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:47:51.485777 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f3310-091e-4812-b918-6bfac095aca4 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:51.536480 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b6f65-7343-4b52-be17-b1658e6393e6 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:51.649038 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f3310-091e-4812-b918-6bfac095aca4 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:51.691025 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:51.717953 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9248708c-14a7-4acd-a57a-f96524f58b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:51.718607 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9248708c-14a7-4acd-a57a-f96524f58b2d None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:51.718891 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9248708c-14a7-4acd-a57a-f96524f58b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:51.719168 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9248708c-14a7-4acd-a57a-f96524f58b2d None admin] PATH_INFO: `/v3/projects/5b12c253c6eb48c9a549a9a8436eced1` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:51.722339 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9248708c-14a7-4acd-a57a-f96524f58b2d None admin] RBAC: Authorizing `identity:get_project(project_id=5b12c253c6eb48c9a549a9a8436eced1)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:51.724188 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9248708c-14a7-4acd-a57a-f96524f58b2d None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:51.725876 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 137/262] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:47:51 2022] GET /identity/v3/projects/5b12c253c6eb48c9a549a9a8436eced1 => generated 302 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:51.731714 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9248708c-14a7-4acd-a57a-f96524f58b2d None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:51.777949 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b6f65-7343-4b52-be17-b1658e6393e6 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:51.863475 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174362e8-8644-4f7b-91af-7ce6123d3b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:51.864791 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174362e8-8644-4f7b-91af-7ce6123d3b28 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:51.865076 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174362e8-8644-4f7b-91af-7ce6123d3b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:51.865354 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174362e8-8644-4f7b-91af-7ce6123d3b28 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:51.891441 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174362e8-8644-4f7b-91af-7ce6123d3b28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:47:51.896487 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-174362e8-8644-4f7b-91af-7ce6123d3b28 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 00:47:51.896487 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:51.896487 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:51.896487 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:51.896487 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:51.896487 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:51.896487 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:51.896487 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:51.896487 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:51.896487 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:51.896487 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:51.896487 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:51.896487 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:47:51.896487 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:47:51.896487 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:47:51.896487 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:47:51.897452 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:47:51.897452 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:47:51.897452 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:47:51.897452 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:47:51.897452 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:51.897452 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:51.897452 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:51.897452 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:51.897452 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:51.897452 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:51.897452 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:51.897452 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:51.897452 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:51.897452 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:51.897452 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:51.897452 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:51.898230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:51.898230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:51.898230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:51.898230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:51.898230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:47:51.898230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:47:51.898230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:47:51.898230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:47:51.898230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:47:51.898230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:47:51.898230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 00:47:51.898230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:47:51.898230 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 126/263] 10.222.0.27 () {56 vars in 1114 bytes} [Sun Sep 4 00:47:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:47:51.905292 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174362e8-8644-4f7b-91af-7ce6123d3b28 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:51.948852 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9248708c-14a7-4acd-a57a-f96524f58b2d None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:52.005130 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9b3e5-3df4-4edb-8b6c-eb7817287e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:52.006221 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9b3e5-3df4-4edb-8b6c-eb7817287e50 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:52.006706 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9b3e5-3df4-4edb-8b6c-eb7817287e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:52.007184 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9b3e5-3df4-4edb-8b6c-eb7817287e50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:52.008536 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf9b3e5-3df4-4edb-8b6c-eb7817287e50 None admin] RBAC: Adding query filter params (role.id=c8a59d34ee2f4cefa6940cac778260c7, scope.project.id=5b12c253c6eb48c9a549a9a8436eced1, user.id=9c6861d39c0949349fb24061ef1c8249) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:52.009058 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf9b3e5-3df4-4edb-8b6c-eb7817287e50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:52.010966 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf9b3e5-3df4-4edb-8b6c-eb7817287e50 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:52.018027 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 138/264] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:47:51 2022] GET /identity/v3/role_assignments?user.id=9c6861d39c0949349fb24061ef1c8249&scope.project.id=5b12c253c6eb48c9a549a9a8436eced1&role.id=c8a59d34ee2f4cefa6940cac778260c7 => generated 608 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:47:52.033766 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:52.127573 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174362e8-8644-4f7b-91af-7ce6123d3b28 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:52.163932 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b25d1f-8a6c-4bd2-980f-906c74f557c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:52.164488 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b25d1f-8a6c-4bd2-980f-906c74f557c2 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:52.164779 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b25d1f-8a6c-4bd2-980f-906c74f557c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:52.165135 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b25d1f-8a6c-4bd2-980f-906c74f557c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:52.165864 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b25d1f-8a6c-4bd2-980f-906c74f557c2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:52.166242 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b25d1f-8a6c-4bd2-980f-906c74f557c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:52.167918 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b25d1f-8a6c-4bd2-980f-906c74f557c2 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:52.177153 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 127/265] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:47:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:47:52.182931 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9b3e5-3df4-4edb-8b6c-eb7817287e50 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:52.262821 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81a38f-a8c9-4e47-bde8-ce33e865e2fc None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:52.263347 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81a38f-a8c9-4e47-bde8-ce33e865e2fc None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:52.263347 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81a38f-a8c9-4e47-bde8-ce33e865e2fc None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:52.265017 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 128/266] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:52.269603 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e66f6e-7ab1-4e96-966d-ac40291db63d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:52.270019 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e66f6e-7ab1-4e96-966d-ac40291db63d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:52.270504 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e66f6e-7ab1-4e96-966d-ac40291db63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:52.271096 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:52.294877 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-30e66f6e-7ab1-4e96-966d-ac40291db63d None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:52.296076 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-30e66f6e-7ab1-4e96-966d-ac40291db63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:52.327227 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9b3e5-3df4-4edb-8b6c-eb7817287e50 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:52.370650 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933ee31d-ba45-4c42-974a-61831d910f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:52.371370 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ee31d-ba45-4c42-974a-61831d910f79 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:52.371701 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ee31d-ba45-4c42-974a-61831d910f79 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:52.371978 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ee31d-ba45-4c42-974a-61831d910f79 None admin] PATH_INFO: `/v3/users/9c6861d39c0949349fb24061ef1c8249` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:52.373160 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933ee31d-ba45-4c42-974a-61831d910f79 None admin] RBAC: Authorizing `identity:get_user(user_id=9c6861d39c0949349fb24061ef1c8249)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:52.374842 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933ee31d-ba45-4c42-974a-61831d910f79 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:52.380823 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 139/267] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:47:52 2022] GET /identity/v3/users/9c6861d39c0949349fb24061ef1c8249 => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:52.382134 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c18bb4-04e5-4e3e-9a66-17bd1a397d93 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:52.382465 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c18bb4-04e5-4e3e-9a66-17bd1a397d93 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:52.382746 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c18bb4-04e5-4e3e-9a66-17bd1a397d93 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:52.384004 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 140/268] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:52.385402 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 129/269] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:52.387164 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c18bb4-04e5-4e3e-9a66-17bd1a397d93 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:52.393063 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca53fe-17bd-4996-87a9-e39824567064 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:52.393348 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca53fe-17bd-4996-87a9-e39824567064 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:52.393626 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca53fe-17bd-4996-87a9-e39824567064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:52.423455 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c9ca53fe-17bd-4996-87a9-e39824567064 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:52.424602 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c9ca53fe-17bd-4996-87a9-e39824567064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:52.465064 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:52.524346 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 130/270] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:52.577148 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c18bb4-04e5-4e3e-9a66-17bd1a397d93 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:52.606845 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:52.623077 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4acfe1-e432-4ef1-a421-d32ce7a76fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:52.623723 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4acfe1-e432-4ef1-a421-d32ce7a76fcc None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:52.624007 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4acfe1-e432-4ef1-a421-d32ce7a76fcc None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:52.624281 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4acfe1-e432-4ef1-a421-d32ce7a76fcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:52.716421 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4acfe1-e432-4ef1-a421-d32ce7a76fcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:52.721922 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4acfe1-e432-4ef1-a421-d32ce7a76fcc None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:52.731298 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca53fe-17bd-4996-87a9-e39824567064 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:52.899177 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 141/271] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:47:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:47:52.900041 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4acfe1-e432-4ef1-a421-d32ce7a76fcc None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:52.901109 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca53fe-17bd-4996-87a9-e39824567064 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:52.938544 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:52.948865 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09302f4b-3913-450e-a32a-90468cc57fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:52.949523 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09302f4b-3913-450e-a32a-90468cc57fd3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:52.949799 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09302f4b-3913-450e-a32a-90468cc57fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:52.950231 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09302f4b-3913-450e-a32a-90468cc57fd3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:52.964717 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09302f4b-3913-450e-a32a-90468cc57fd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:47:52.971125 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-09302f4b-3913-450e-a32a-90468cc57fd3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:47:52.971125 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:52.971125 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:52.971125 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:52.971125 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:52.971125 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:52.971125 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:52.971125 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:52.971125 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:52.971125 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:52.971125 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:52.971125 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:52.971125 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:47:52.971125 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:47:52.971125 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:47:52.971125 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:47:52.973337 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:47:52.973337 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:47:52.973337 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:47:52.973337 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:47:52.973337 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:52.973337 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:52.973337 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:52.973337 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:52.973337 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:52.973337 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:52.973337 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:52.973337 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:52.973337 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:52.973337 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:52.973337 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:52.973337 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:52.975310 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:52.975310 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:52.975310 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:52.975310 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:52.975310 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:47:52.975310 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:47:52.975310 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:47:52.975310 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:47:52.975310 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:47:52.975310 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:47:52.975310 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:47:52.975310 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:47:52.975310 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 131/272] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:47:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:47:52.978087 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09302f4b-3913-450e-a32a-90468cc57fd3 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:53.010334 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:53.037065 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4acfe1-e432-4ef1-a421-d32ce7a76fcc None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:53.075963 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e230026-b6c1-4c2a-8508-19b11fe9c2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:53.076724 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e230026-b6c1-4c2a-8508-19b11fe9c2c0 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:53.077008 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e230026-b6c1-4c2a-8508-19b11fe9c2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:53.077277 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e230026-b6c1-4c2a-8508-19b11fe9c2c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:53.078389 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e230026-b6c1-4c2a-8508-19b11fe9c2c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:53.080063 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e230026-b6c1-4c2a-8508-19b11fe9c2c0 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:53.082023 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 142/273] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:47:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:53.083116 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e230026-b6c1-4c2a-8508-19b11fe9c2c0 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:53.085511 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09302f4b-3913-450e-a32a-90468cc57fd3 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:53.124660 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8686e3ca-4829-46da-bfe8-764ad40d1a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:53.125354 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686e3ca-4829-46da-bfe8-764ad40d1a1b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:53.125593 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686e3ca-4829-46da-bfe8-764ad40d1a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:53.125866 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686e3ca-4829-46da-bfe8-764ad40d1a1b None admin] PATH_INFO: `/v3/projects/5b12c253c6eb48c9a549a9a8436eced1` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:53.127033 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8686e3ca-4829-46da-bfe8-764ad40d1a1b None admin] RBAC: Authorizing `identity:get_project(project_id=5b12c253c6eb48c9a549a9a8436eced1)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:53.128569 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8686e3ca-4829-46da-bfe8-764ad40d1a1b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:53.131199 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 132/274] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:47:52 2022] GET /identity/v3/projects/5b12c253c6eb48c9a549a9a8436eced1 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:53.132122 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8686e3ca-4829-46da-bfe8-764ad40d1a1b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:53.188171 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:53.198357 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e230026-b6c1-4c2a-8508-19b11fe9c2c0 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:53.235898 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:53.237027 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf4f7c-179a-49d1-be33-ef823c6f3ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:53.237852 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf4f7c-179a-49d1-be33-ef823c6f3ae0 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:53.238278 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf4f7c-179a-49d1-be33-ef823c6f3ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:53.239109 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf4f7c-179a-49d1-be33-ef823c6f3ae0 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:53.239923 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf4f7c-179a-49d1-be33-ef823c6f3ae0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:53.240227 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf4f7c-179a-49d1-be33-ef823c6f3ae0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:53.241641 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf4f7c-179a-49d1-be33-ef823c6f3ae0 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:53.242459 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8686e3ca-4829-46da-bfe8-764ad40d1a1b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:53.251130 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 143/275] 10.222.0.27 () {56 vars in 1101 bytes} [Sun Sep 4 00:47:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:53.257613 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a10818-1be5-4981-9dcd-3a1507cbc0af None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:53.257897 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a10818-1be5-4981-9dcd-3a1507cbc0af None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:53.258170 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a10818-1be5-4981-9dcd-3a1507cbc0af None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:53.259562 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 144/276] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:53.260124 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a10818-1be5-4981-9dcd-3a1507cbc0af None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:53.282603 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b7cf8-328e-4561-82bd-eba5aa6ce728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:53.283204 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b7cf8-328e-4561-82bd-eba5aa6ce728 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:53.283662 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b7cf8-328e-4561-82bd-eba5aa6ce728 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:53.284089 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b7cf8-328e-4561-82bd-eba5aa6ce728 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:53.284776 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b7cf8-328e-4561-82bd-eba5aa6ce728 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:53.286407 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b7cf8-328e-4561-82bd-eba5aa6ce728 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:53.366042 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a10818-1be5-4981-9dcd-3a1507cbc0af None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:53.395245 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 133/277] 10.222.0.27 () {60 vars in 1132 bytes} [Sun Sep 4 00:47:53 2022] POST /identity/v3/users => generated 293 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:47:53.397415 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b7cf8-328e-4561-82bd-eba5aa6ce728 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:53.417195 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed60477-2b32-4a6d-9233-328ae9a5abcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:53.418509 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed60477-2b32-4a6d-9233-328ae9a5abcf None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:53.419063 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed60477-2b32-4a6d-9233-328ae9a5abcf None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:53.419584 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed60477-2b32-4a6d-9233-328ae9a5abcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:53.420606 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed60477-2b32-4a6d-9233-328ae9a5abcf None admin] RBAC: Adding query filter params (role.id=7df9ebc56a2a4214b2b64a5941812ef0, scope.project.id=5b12c253c6eb48c9a549a9a8436eced1, user.id=9c6861d39c0949349fb24061ef1c8249) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:53.421303 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed60477-2b32-4a6d-9233-328ae9a5abcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:53.423219 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed60477-2b32-4a6d-9233-328ae9a5abcf None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:53.432100 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 145/278] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:47:53 2022] GET /identity/v3/role_assignments?user.id=9c6861d39c0949349fb24061ef1c8249&scope.project.id=5b12c253c6eb48c9a549a9a8436eced1&role.id=7df9ebc56a2a4214b2b64a5941812ef0 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:53.433890 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2173c-564f-4108-864d-00dd28e33786 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:53.434412 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2173c-564f-4108-864d-00dd28e33786 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:53.434953 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2173c-564f-4108-864d-00dd28e33786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:53.466591 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-94e2173c-564f-4108-864d-00dd28e33786 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:53.474094 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-94e2173c-564f-4108-864d-00dd28e33786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:53.506516 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:53.577918 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 146/279] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:53.579084 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8a7e8-31ef-48ba-9a87-4df636bae068 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:53.579415 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8a7e8-31ef-48ba-9a87-4df636bae068 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:53.579632 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8a7e8-31ef-48ba-9a87-4df636bae068 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:53.581026 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 147/280] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:53.583508 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b7cf8-328e-4561-82bd-eba5aa6ce728 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:53.587418 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e2218-f57e-443f-978a-388eaace8f76 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:53.587709 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e2218-f57e-443f-978a-388eaace8f76 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:53.587992 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e2218-f57e-443f-978a-388eaace8f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:53.608638 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-820e2218-f57e-443f-978a-388eaace8f76 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:53.609975 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-820e2218-f57e-443f-978a-388eaace8f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:53.628427 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456a379-f1d4-4d77-b503-fd9800e696a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:53.630144 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456a379-f1d4-4d77-b503-fd9800e696a5 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:53.630967 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456a379-f1d4-4d77-b503-fd9800e696a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:53.631468 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456a379-f1d4-4d77-b503-fd9800e696a5 None admin] PATH_INFO: `/v3/users/b978689a22cc407ea316eba38b779314` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:53.651720 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c456a379-f1d4-4d77-b503-fd9800e696a5 None admin] RBAC: Authorizing `identity:get_user(user_id=b978689a22cc407ea316eba38b779314)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:53.653293 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c456a379-f1d4-4d77-b503-fd9800e696a5 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:53.654977 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 134/281] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:47:53 2022] GET /identity/v3/users/b978689a22cc407ea316eba38b779314 => generated 293 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:53.661675 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456a379-f1d4-4d77-b503-fd9800e696a5 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:53.677218 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:53.692415 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 148/282] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 00:47:53.783300 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456a379-f1d4-4d77-b503-fd9800e696a5 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:53.821923 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca48fb3e-758f-4fb2-81c6-65792b768e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:53.822533 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48fb3e-758f-4fb2-81c6-65792b768e0d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:53.822867 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48fb3e-758f-4fb2-81c6-65792b768e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:53.823150 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48fb3e-758f-4fb2-81c6-65792b768e0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:53.823877 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca48fb3e-758f-4fb2-81c6-65792b768e0d None admin] RBAC: Adding query filter params (role.id=d1261c99e1d64f9da63adfff739561e8, user.id=b978689a22cc407ea316eba38b779314, scope.system=all) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:53.824187 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca48fb3e-758f-4fb2-81c6-65792b768e0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:53.825663 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca48fb3e-758f-4fb2-81c6-65792b768e0d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:53.831244 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 135/283] 10.222.0.27 () {56 vars in 1308 bytes} [Sun Sep 4 00:47:53 2022] GET /identity/v3/role_assignments?user.id=b978689a22cc407ea316eba38b779314&scope.system=all&role.id=d1261c99e1d64f9da63adfff739561e8 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:53.883935 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e2218-f57e-443f-978a-388eaace8f76 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:53.984188 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca48fb3e-758f-4fb2-81c6-65792b768e0d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:53.990757 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:54.017070 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e2218-f57e-443f-978a-388eaace8f76 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:54.061451 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc9ecd9-cc8f-433a-9663-f0578fef7239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:54.062422 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9ecd9-cc8f-433a-9663-f0578fef7239 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:54.062714 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9ecd9-cc8f-433a-9663-f0578fef7239 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:54.062992 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9ecd9-cc8f-433a-9663-f0578fef7239 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:54.077032 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc9ecd9-cc8f-433a-9663-f0578fef7239 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:47:54.078575 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:54.085320 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-5dc9ecd9-cc8f-433a-9663-f0578fef7239 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 00:47:54.085320 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:54.085320 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:54.085320 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:54.085320 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:54.085320 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:54.085320 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:54.085320 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:54.085320 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:54.085320 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:54.085320 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:54.085320 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:54.085320 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:47:54.085320 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:47:54.085320 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:47:54.085320 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:47:54.092690 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:47:54.092690 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:47:54.092690 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:47:54.092690 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:47:54.092690 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:54.092690 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:54.092690 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:54.092690 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:54.092690 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:54.092690 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:54.092690 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:54.092690 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:54.092690 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:54.092690 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:54.092690 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:54.092690 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:54.093576 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:54.093576 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:54.093576 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:54.093576 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:54.093576 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:47:54.093576 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:47:54.093576 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:47:54.093576 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:47:54.093576 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:47:54.093576 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:47:54.093576 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 00:47:54.093576 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:47:54.093576 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 149/284] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:47:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:47:54.094213 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc9ecd9-cc8f-433a-9663-f0578fef7239 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:54.124370 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca48fb3e-758f-4fb2-81c6-65792b768e0d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:54.193970 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e6610d-df58-4814-aad3-5351a703f5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:54.195313 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e6610d-df58-4814-aad3-5351a703f5f1 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:54.196474 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e6610d-df58-4814-aad3-5351a703f5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:54.197148 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e6610d-df58-4814-aad3-5351a703f5f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:54.220332 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e6610d-df58-4814-aad3-5351a703f5f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:47:54.227647 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-b5e6610d-df58-4814-aad3-5351a703f5f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:47:54.227647 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:54.227647 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:54.227647 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:54.227647 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:54.227647 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:54.227647 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:54.227647 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:54.227647 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:54.227647 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:54.227647 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:54.227647 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:54.227647 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:47:54.227647 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:47:54.227647 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:47:54.227647 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:47:54.228561 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:47:54.228561 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:47:54.228561 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:47:54.228561 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:47:54.228561 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:54.228561 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:54.228561 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:54.228561 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:54.228561 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:54.228561 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:54.228561 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:54.228561 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:54.228561 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:54.228561 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:54.228561 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:54.228561 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:54.229341 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:54.229341 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:54.229341 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:54.229341 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:54.229341 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:47:54.229341 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:47:54.229341 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:47:54.229341 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:47:54.229341 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:47:54.229341 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:47:54.229341 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:47:54.229341 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:47:54.235377 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 136/285] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:47:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:47:54.245381 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e6610d-df58-4814-aad3-5351a703f5f1 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:54.306039 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc9ecd9-cc8f-433a-9663-f0578fef7239 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:54.359578 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618541b4-506e-4e41-8e2e-f28710ba931e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:54.360152 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618541b4-506e-4e41-8e2e-f28710ba931e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:54.360431 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618541b4-506e-4e41-8e2e-f28710ba931e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:54.360722 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618541b4-506e-4e41-8e2e-f28710ba931e None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:54.362124 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618541b4-506e-4e41-8e2e-f28710ba931e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:54.362426 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618541b4-506e-4e41-8e2e-f28710ba931e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:54.364009 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618541b4-506e-4e41-8e2e-f28710ba931e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:54.375060 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e6610d-df58-4814-aad3-5351a703f5f1 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:54.377017 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:54.382928 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 150/286] 10.222.0.27 () {56 vars in 1101 bytes} [Sun Sep 4 00:47:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:54.388832 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618541b4-506e-4e41-8e2e-f28710ba931e None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:54.451235 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df82e36e-7a10-4c09-b272-c92b39a8fdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:54.452211 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82e36e-7a10-4c09-b272-c92b39a8fdcb None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:54.452741 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82e36e-7a10-4c09-b272-c92b39a8fdcb None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:54.453234 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82e36e-7a10-4c09-b272-c92b39a8fdcb None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:54.454096 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df82e36e-7a10-4c09-b272-c92b39a8fdcb None admin] RBAC: Adding query filter params (name=member) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:54.454595 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df82e36e-7a10-4c09-b272-c92b39a8fdcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:54.456179 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df82e36e-7a10-4c09-b272-c92b39a8fdcb None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:54.459903 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:54.475614 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 137/287] 10.222.0.27 () {56 vars in 1101 bytes} [Sun Sep 4 00:47:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:54.482008 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df82e36e-7a10-4c09-b272-c92b39a8fdcb None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:54.564587 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618541b4-506e-4e41-8e2e-f28710ba931e None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:54.620011 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df82e36e-7a10-4c09-b272-c92b39a8fdcb None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:54.635665 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed2a4f-082b-468f-89ee-7805a2f4d76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:54.636666 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed2a4f-082b-468f-89ee-7805a2f4d76a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:54.637151 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed2a4f-082b-468f-89ee-7805a2f4d76a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:54.637626 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed2a4f-082b-468f-89ee-7805a2f4d76a None admin] PATH_INFO: `/v3/users/d11f31e750454372b47b35c41fcc9726` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:54.639357 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed2a4f-082b-468f-89ee-7805a2f4d76a None admin] RBAC: Authorizing `identity:get_user(user_id=d11f31e750454372b47b35c41fcc9726)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:54.641573 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed2a4f-082b-468f-89ee-7805a2f4d76a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:54.643539 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 151/288] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:47:54 2022] GET /identity/v3/users/d11f31e750454372b47b35c41fcc9726 => generated 297 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:47:54.650037 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed2a4f-082b-468f-89ee-7805a2f4d76a None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:54.685584 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d60cee-bb7f-4427-9c52-c5cafb435361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:54.686219 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d60cee-bb7f-4427-9c52-c5cafb435361 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:54.686527 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d60cee-bb7f-4427-9c52-c5cafb435361 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:54.686778 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d60cee-bb7f-4427-9c52-c5cafb435361 None admin] PATH_INFO: `/v3/users/e133dde4c9cb4debbefdd320b783c004` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:54.687891 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d60cee-bb7f-4427-9c52-c5cafb435361 None admin] RBAC: Authorizing `identity:get_user(user_id=e133dde4c9cb4debbefdd320b783c004)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:54.693485 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d60cee-bb7f-4427-9c52-c5cafb435361 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:54.695261 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 138/289] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:47:54 2022] GET /identity/v3/users/e133dde4c9cb4debbefdd320b783c004 => generated 297 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:54.702662 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d60cee-bb7f-4427-9c52-c5cafb435361 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:54.730149 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:54.769329 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed2a4f-082b-468f-89ee-7805a2f4d76a None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:54.827153 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:54.835985 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f62774-8755-4ecd-a7eb-91621b7c5764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:54.836659 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f62774-8755-4ecd-a7eb-91621b7c5764 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:54.836937 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f62774-8755-4ecd-a7eb-91621b7c5764 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:54.837206 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f62774-8755-4ecd-a7eb-91621b7c5764 None admin] PATH_INFO: `/v3/projects/5b12c253c6eb48c9a549a9a8436eced1` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:54.840820 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f62774-8755-4ecd-a7eb-91621b7c5764 None admin] RBAC: Authorizing `identity:get_project(project_id=5b12c253c6eb48c9a549a9a8436eced1)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:54.843988 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f62774-8755-4ecd-a7eb-91621b7c5764 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:54.845660 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 152/290] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:47:54 2022] GET /identity/v3/projects/5b12c253c6eb48c9a549a9a8436eced1 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:47:54.852136 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f62774-8755-4ecd-a7eb-91621b7c5764 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:54.891692 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d60cee-bb7f-4427-9c52-c5cafb435361 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:54.929421 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb64d6b-c2fb-440d-9922-b44f1bf7366e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:54.930024 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb64d6b-c2fb-440d-9922-b44f1bf7366e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:54.930303 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb64d6b-c2fb-440d-9922-b44f1bf7366e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:54.930941 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb64d6b-c2fb-440d-9922-b44f1bf7366e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:55.030640 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb64d6b-c2fb-440d-9922-b44f1bf7366e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:55.032332 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb64d6b-c2fb-440d-9922-b44f1bf7366e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:55.115018 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f62774-8755-4ecd-a7eb-91621b7c5764 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:55.120807 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:55.139915 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 139/291] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:47:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 04 00:47:55.148733 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb64d6b-c2fb-440d-9922-b44f1bf7366e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:55.201153 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c3171-2f63-4ba3-93df-e340aff33967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:55.202065 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c3171-2f63-4ba3-93df-e340aff33967 None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:55.202491 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c3171-2f63-4ba3-93df-e340aff33967 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:55.202908 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c3171-2f63-4ba3-93df-e340aff33967 None admin] PATH_INFO: `/v3/projects/5b12c253c6eb48c9a549a9a8436eced1/users/d11f31e750454372b47b35c41fcc9726/roles/8f7a2ac695c6482cbf48819cee250c22` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:55.205058 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c3171-2f63-4ba3-93df-e340aff33967 None admin] RBAC: Authorizing `identity:create_grant(project_id=5b12c253c6eb48c9a549a9a8436eced1, user_id=d11f31e750454372b47b35c41fcc9726, role_id=8f7a2ac695c6482cbf48819cee250c22)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:55.206651 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c3171-2f63-4ba3-93df-e340aff33967 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:55.211014 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:55.257779 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb64d6b-c2fb-440d-9922-b44f1bf7366e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:55.284204 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 153/292] 10.222.0.27 () {58 vars in 1439 bytes} [Sun Sep 4 00:47:54 2022] PUT /identity/v3/projects/5b12c253c6eb48c9a549a9a8436eced1/users/d11f31e750454372b47b35c41fcc9726/roles/8f7a2ac695c6482cbf48819cee250c22 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 00:47:55.294138 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de468a-3059-45f8-83a4-3c7a95e39a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:55.294788 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de468a-3059-45f8-83a4-3c7a95e39a88 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:55.295067 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de468a-3059-45f8-83a4-3c7a95e39a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:55.295340 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de468a-3059-45f8-83a4-3c7a95e39a88 None admin] PATH_INFO: `/v3/projects/5b12c253c6eb48c9a549a9a8436eced1` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:55.296589 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de468a-3059-45f8-83a4-3c7a95e39a88 None admin] RBAC: Authorizing `identity:get_project(project_id=5b12c253c6eb48c9a549a9a8436eced1)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:55.298152 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de468a-3059-45f8-83a4-3c7a95e39a88 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:55.300306 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 140/293] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:47:55 2022] GET /identity/v3/projects/5b12c253c6eb48c9a549a9a8436eced1 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:55.307559 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c3171-2f63-4ba3-93df-e340aff33967 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:55.471608 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c3171-2f63-4ba3-93df-e340aff33967 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:55.513511 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a82ae-fd80-4609-ad38-e4907d80a1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:55.514130 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a82ae-fd80-4609-ad38-e4907d80a1dc None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:55.514413 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a82ae-fd80-4609-ad38-e4907d80a1dc None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:55.514692 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a82ae-fd80-4609-ad38-e4907d80a1dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:55.515426 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a82ae-fd80-4609-ad38-e4907d80a1dc None admin] RBAC: Adding query filter params (role.id=d1261c99e1d64f9da63adfff739561e8, scope.project.id=5b12c253c6eb48c9a549a9a8436eced1, user.id=e133dde4c9cb4debbefdd320b783c004) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:55.515736 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a82ae-fd80-4609-ad38-e4907d80a1dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:55.517290 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a82ae-fd80-4609-ad38-e4907d80a1dc None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:55.525116 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 154/294] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:47:55 2022] GET /identity/v3/role_assignments?user.id=e133dde4c9cb4debbefdd320b783c004&scope.project.id=5b12c253c6eb48c9a549a9a8436eced1&role.id=d1261c99e1d64f9da63adfff739561e8 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:55.990001 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b62c8-8a97-4ba4-8441-ff699156326a None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:55.990517 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b62c8-8a97-4ba4-8441-ff699156326a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:55.990517 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b62c8-8a97-4ba4-8441-ff699156326a None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:55.991867 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 141/295] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:55.997221 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41956bb2-3c01-4461-8e82-2b973bde867a None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:55.997666 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41956bb2-3c01-4461-8e82-2b973bde867a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:55.998078 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41956bb2-3c01-4461-8e82-2b973bde867a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:56.019399 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-41956bb2-3c01-4461-8e82-2b973bde867a None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:56.021411 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-41956bb2-3c01-4461-8e82-2b973bde867a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:56.082599 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff271d86-5cec-429e-9a06-e81bb173f6c8 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:56.083731 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff271d86-5cec-429e-9a06-e81bb173f6c8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:56.084155 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff271d86-5cec-429e-9a06-e81bb173f6c8 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:56.085924 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 142/296] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:56.086292 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:56.093453 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3b989-e0d6-4c88-b6cd-3e8fe1403cdc None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:56.094206 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3b989-e0d6-4c88-b6cd-3e8fe1403cdc None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:56.094794 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3b989-e0d6-4c88-b6cd-3e8fe1403cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:56.103822 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 155/297] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:56.117980 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-fbc3b989-e0d6-4c88-b6cd-3e8fe1403cdc None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:56.119181 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-fbc3b989-e0d6-4c88-b6cd-3e8fe1403cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:56.147366 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:56.194777 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 143/298] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:56.197786 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfcd409-2092-42fc-973e-39b57910e66b None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:56.198075 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfcd409-2092-42fc-973e-39b57910e66b None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:56.198354 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfcd409-2092-42fc-973e-39b57910e66b None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:56.199678 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 156/299] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:56.204800 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09411f78-7739-4e0f-a674-1e9936caa713 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:56.205097 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09411f78-7739-4e0f-a674-1e9936caa713 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:56.205377 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09411f78-7739-4e0f-a674-1e9936caa713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:56.224596 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-09411f78-7739-4e0f-a674-1e9936caa713 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:56.225804 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-09411f78-7739-4e0f-a674-1e9936caa713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:56.295050 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 144/300] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:56.401376 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfcd409-2092-42fc-973e-39b57910e66b None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:56.499026 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09411f78-7739-4e0f-a674-1e9936caa713 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:56.514031 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfcd409-2092-42fc-973e-39b57910e66b None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:56.553471 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e3834-e020-4845-a086-9fe662dc5c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:56.554034 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e3834-e020-4845-a086-9fe662dc5c3e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:56.554312 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e3834-e020-4845-a086-9fe662dc5c3e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:56.554589 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e3834-e020-4845-a086-9fe662dc5c3e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:56.565689 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e3834-e020-4845-a086-9fe662dc5c3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:47:56.571512 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-815e3834-e020-4845-a086-9fe662dc5c3e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:47:56.571512 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:56.571512 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:56.571512 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:56.571512 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:56.571512 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:56.571512 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:56.571512 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:56.571512 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:56.571512 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:56.571512 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:56.571512 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:56.571512 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:47:56.571512 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:47:56.571512 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:47:56.571512 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:47:56.572413 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:47:56.572413 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:47:56.572413 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:47:56.572413 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:47:56.572413 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:56.572413 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:56.572413 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:56.572413 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:56.572413 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:56.572413 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:56.572413 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:56.572413 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:56.572413 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:56.572413 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:56.572413 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:56.572413 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:56.573653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:56.573653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:56.573653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:56.573653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:56.573653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:47:56.573653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:47:56.573653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:47:56.573653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:47:56.573653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:47:56.573653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:47:56.573653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:47:56.573653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:47:56.573653 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 157/301] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:47:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:47:56.578752 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e3834-e020-4845-a086-9fe662dc5c3e None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:56.599801 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:56.610515 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09411f78-7739-4e0f-a674-1e9936caa713 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:56.635679 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:56.650619 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e84347-cb35-44d0-9d7e-10bccfe8a621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:56.651494 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e84347-cb35-44d0-9d7e-10bccfe8a621 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:56.651959 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e84347-cb35-44d0-9d7e-10bccfe8a621 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:56.652414 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e84347-cb35-44d0-9d7e-10bccfe8a621 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:56.663248 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e84347-cb35-44d0-9d7e-10bccfe8a621 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:47:56.668277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-38e84347-cb35-44d0-9d7e-10bccfe8a621 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 00:47:56.668277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:56.668277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:56.668277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:56.668277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:56.668277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:56.668277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:56.668277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:56.668277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:56.668277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:56.668277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:56.668277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:56.668277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:47:56.668277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:47:56.668277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:47:56.668277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:47:56.669177 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:47:56.669177 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:47:56.669177 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:47:56.669177 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:47:56.669177 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:56.669177 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:56.669177 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:56.669177 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:56.669177 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:56.669177 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:56.669177 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:56.669177 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:56.669177 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:56.669177 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:56.669177 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:56.669177 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:56.669942 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:56.669942 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:56.669942 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:56.669942 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:56.669942 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:47:56.669942 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:47:56.669942 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:47:56.669942 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:47:56.669942 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:47:56.669942 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:47:56.669942 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 00:47:56.669942 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:47:56.669942 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 145/302] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:47:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:47:56.670616 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e84347-cb35-44d0-9d7e-10bccfe8a621 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:56.694555 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e3834-e020-4845-a086-9fe662dc5c3e None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:56.730391 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b0b97d-58da-4116-aae8-fc2bc4f1e47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:56.730989 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0b97d-58da-4116-aae8-fc2bc4f1e47a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:56.731267 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0b97d-58da-4116-aae8-fc2bc4f1e47a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:56.731540 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0b97d-58da-4116-aae8-fc2bc4f1e47a None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:56.732191 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b0b97d-58da-4116-aae8-fc2bc4f1e47a None admin] RBAC: Adding query filter params (name=member) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:56.732516 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b0b97d-58da-4116-aae8-fc2bc4f1e47a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:56.733888 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b0b97d-58da-4116-aae8-fc2bc4f1e47a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:56.742677 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 158/303] 10.222.0.27 () {56 vars in 1101 bytes} [Sun Sep 4 00:47:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:56.744620 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b0b97d-58da-4116-aae8-fc2bc4f1e47a None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:56.784261 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e84347-cb35-44d0-9d7e-10bccfe8a621 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:56.823422 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d10de-0a8d-4630-9a4e-7b688f137b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:56.823973 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d10de-0a8d-4630-9a4e-7b688f137b18 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:56.824264 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d10de-0a8d-4630-9a4e-7b688f137b18 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:56.824569 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d10de-0a8d-4630-9a4e-7b688f137b18 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:56.825686 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d10de-0a8d-4630-9a4e-7b688f137b18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:56.827134 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d10de-0a8d-4630-9a4e-7b688f137b18 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:56.828729 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 146/304] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:47:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:56.829493 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d10de-0a8d-4630-9a4e-7b688f137b18 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:56.852886 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:56.856894 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b0b97d-58da-4116-aae8-fc2bc4f1e47a None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:56.871043 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:56.894408 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193b27a5-4076-499d-8dc5-eea09517ad02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:56.895124 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b27a5-4076-499d-8dc5-eea09517ad02 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:56.895416 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b27a5-4076-499d-8dc5-eea09517ad02 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:56.895695 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b27a5-4076-499d-8dc5-eea09517ad02 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:56.896366 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193b27a5-4076-499d-8dc5-eea09517ad02 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:56.896703 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193b27a5-4076-499d-8dc5-eea09517ad02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:56.898087 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193b27a5-4076-499d-8dc5-eea09517ad02 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:56.911341 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 159/305] 10.222.0.27 () {56 vars in 1101 bytes} [Sun Sep 4 00:47:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:56.911341 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193b27a5-4076-499d-8dc5-eea09517ad02 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:56.946669 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d10de-0a8d-4630-9a4e-7b688f137b18 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:56.982988 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b4c89b-3a87-45f8-98fb-cf62f730d2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:56.983662 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4c89b-3a87-45f8-98fb-cf62f730d2bb None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:56.983882 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4c89b-3a87-45f8-98fb-cf62f730d2bb None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:56.984155 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4c89b-3a87-45f8-98fb-cf62f730d2bb None admin] PATH_INFO: `/v3/users/b978689a22cc407ea316eba38b779314` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:56.985298 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b4c89b-3a87-45f8-98fb-cf62f730d2bb None admin] RBAC: Authorizing `identity:get_user(user_id=b978689a22cc407ea316eba38b779314)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:56.986736 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b4c89b-3a87-45f8-98fb-cf62f730d2bb None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:56.989021 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 147/306] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:47:56 2022] GET /identity/v3/users/b978689a22cc407ea316eba38b779314 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:56.990102 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b4c89b-3a87-45f8-98fb-cf62f730d2bb None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:57.022504 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193b27a5-4076-499d-8dc5-eea09517ad02 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:57.059297 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40660b4-120e-4465-aad8-9152b58b0f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:57.059899 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40660b4-120e-4465-aad8-9152b58b0f94 None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:57.060175 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40660b4-120e-4465-aad8-9152b58b0f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:57.060444 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40660b4-120e-4465-aad8-9152b58b0f94 None admin] PATH_INFO: `/v3/groups` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:57.061112 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40660b4-120e-4465-aad8-9152b58b0f94 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:57.062762 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40660b4-120e-4465-aad8-9152b58b0f94 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:57.084122 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:57.095483 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b4c89b-3a87-45f8-98fb-cf62f730d2bb None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:57.129810 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c2345-f987-4a40-9851-5c198cfb8a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:57.129810 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c2345-f987-4a40-9851-5c198cfb8a1f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:57.129810 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c2345-f987-4a40-9851-5c198cfb8a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:57.130355 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c2345-f987-4a40-9851-5c198cfb8a1f None admin] PATH_INFO: `/v3/users/0ce42cd5938040f1a4f5925b0238dd68` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:57.148488 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 160/307] 10.222.0.27 () {60 vars in 1134 bytes} [Sun Sep 4 00:47:56 2022] POST /identity/v3/groups => generated 229 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:47:57.148488 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40660b4-120e-4465-aad8-9152b58b0f94 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:57.161154 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c2345-f987-4a40-9851-5c198cfb8a1f None admin] RBAC: Authorizing `identity:get_user(user_id=0ce42cd5938040f1a4f5925b0238dd68)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:57.163102 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c2345-f987-4a40-9851-5c198cfb8a1f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:57.165263 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 148/308] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:47:56 2022] GET /identity/v3/users/0ce42cd5938040f1a4f5925b0238dd68 => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:57.171906 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c2345-f987-4a40-9851-5c198cfb8a1f None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:57.188843 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:57.279763 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40660b4-120e-4465-aad8-9152b58b0f94 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:57.298554 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c2345-f987-4a40-9851-5c198cfb8a1f None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:57.319934 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23298feb-7ea6-4112-b8b9-7ff52dc8278e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:57.321268 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23298feb-7ea6-4112-b8b9-7ff52dc8278e None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:57.321268 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23298feb-7ea6-4112-b8b9-7ff52dc8278e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:57.321268 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23298feb-7ea6-4112-b8b9-7ff52dc8278e None admin] PATH_INFO: `/v3/system/users/b978689a22cc407ea316eba38b779314/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:57.327941 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23298feb-7ea6-4112-b8b9-7ff52dc8278e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b978689a22cc407ea316eba38b779314, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:57.327941 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23298feb-7ea6-4112-b8b9-7ff52dc8278e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:57.340413 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08e101-2233-4c11-ae6c-fe02ef51707e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:57.340413 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08e101-2233-4c11-ae6c-fe02ef51707e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:57.340413 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08e101-2233-4c11-ae6c-fe02ef51707e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:57.340413 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08e101-2233-4c11-ae6c-fe02ef51707e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:57.340791 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08e101-2233-4c11-ae6c-fe02ef51707e None admin] RBAC: Adding query filter params (role.id=8f7a2ac695c6482cbf48819cee250c22, user.id=0ce42cd5938040f1a4f5925b0238dd68, scope.system=all) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:57.340791 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08e101-2233-4c11-ae6c-fe02ef51707e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:57.340949 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08e101-2233-4c11-ae6c-fe02ef51707e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:57.346261 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 149/309] 10.222.0.27 () {56 vars in 1308 bytes} [Sun Sep 4 00:47:57 2022] GET /identity/v3/role_assignments?user.id=0ce42cd5938040f1a4f5925b0238dd68&scope.system=all&role.id=8f7a2ac695c6482cbf48819cee250c22 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:47:57.419235 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 161/310] 10.222.0.27 () {58 vars in 1334 bytes} [Sun Sep 4 00:47:57 2022] PUT /identity/v3/system/users/b978689a22cc407ea316eba38b779314/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 00:47:57.475518 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6eb900-d339-4e5c-a2bd-417a6ec459c9 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:57.475872 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6eb900-d339-4e5c-a2bd-417a6ec459c9 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:57.476080 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6eb900-d339-4e5c-a2bd-417a6ec459c9 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:57.477591 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 150/311] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:57.483918 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d512e-055c-487f-85bf-5d70b68116cd None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:57.484217 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d512e-055c-487f-85bf-5d70b68116cd None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:57.484513 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d512e-055c-487f-85bf-5d70b68116cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:57.506666 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-140d512e-055c-487f-85bf-5d70b68116cd None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:57.508047 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-140d512e-055c-487f-85bf-5d70b68116cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:57.588172 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 162/312] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:57.861720 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6eb900-d339-4e5c-a2bd-417a6ec459c9 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:57.875271 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:57.976744 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6eb900-d339-4e5c-a2bd-417a6ec459c9 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:58.037688 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeb2cc4-b675-417d-9090-46788c65ecc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:58.038361 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb2cc4-b675-417d-9090-46788c65ecc6 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:58.038648 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb2cc4-b675-417d-9090-46788c65ecc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:58.038924 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb2cc4-b675-417d-9090-46788c65ecc6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:58.049814 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeb2cc4-b675-417d-9090-46788c65ecc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:47:58.055517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-dfeb2cc4-b675-417d-9090-46788c65ecc6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 00:47:58.055517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:58.055517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:58.055517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:58.055517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:58.055517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:58.055517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:58.055517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:58.055517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:58.055517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:58.055517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:58.055517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:58.055517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:47:58.055517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:47:58.055517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:47:58.055517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:47:58.056673 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:47:58.056673 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:47:58.056673 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:47:58.056673 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:47:58.056673 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:58.056673 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:58.056673 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:58.056673 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:58.056673 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:58.056673 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:58.056673 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:58.056673 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:58.056673 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:58.056673 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:58.056673 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:58.056673 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:58.057448 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:58.057448 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:58.057448 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:58.057448 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:58.057448 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:47:58.057448 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:47:58.057448 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:47:58.057448 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:47:58.057448 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:47:58.057448 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:47:58.057448 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 00:47:58.057448 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:47:58.057448 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 151/313] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:47:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:47:58.064438 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d512e-055c-487f-85bf-5d70b68116cd None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:58.093858 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:58.169454 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d512e-055c-487f-85bf-5d70b68116cd None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:58.204739 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a42c9ab-0acc-4bcd-9f1d-6375e180d515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:58.205344 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a42c9ab-0acc-4bcd-9f1d-6375e180d515 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:58.205622 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a42c9ab-0acc-4bcd-9f1d-6375e180d515 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:58.205894 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a42c9ab-0acc-4bcd-9f1d-6375e180d515 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:58.206551 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a42c9ab-0acc-4bcd-9f1d-6375e180d515 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:58.206851 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a42c9ab-0acc-4bcd-9f1d-6375e180d515 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:58.208959 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a42c9ab-0acc-4bcd-9f1d-6375e180d515 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:58.217447 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 163/314] 10.222.0.27 () {56 vars in 1101 bytes} [Sun Sep 4 00:47:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:58.225885 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeb2cc4-b675-417d-9090-46788c65ecc6 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:58.304684 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:58.341242 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeb2cc4-b675-417d-9090-46788c65ecc6 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:58.376729 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759c433-d960-403e-a933-7c35079c912b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:58.377363 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759c433-d960-403e-a933-7c35079c912b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:58.377645 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759c433-d960-403e-a933-7c35079c912b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:58.377920 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759c433-d960-403e-a933-7c35079c912b None admin] PATH_INFO: `/v3/users/d11f31e750454372b47b35c41fcc9726` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:58.379628 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6759c433-d960-403e-a933-7c35079c912b None admin] RBAC: Authorizing `identity:get_user(user_id=d11f31e750454372b47b35c41fcc9726)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:58.381114 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6759c433-d960-403e-a933-7c35079c912b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:58.383071 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 152/315] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:47:58 2022] GET /identity/v3/users/d11f31e750454372b47b35c41fcc9726 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:58.389772 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a42c9ab-0acc-4bcd-9f1d-6375e180d515 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:58.490339 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:58.500148 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a42c9ab-0acc-4bcd-9f1d-6375e180d515 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:58.538472 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9d2434-1100-4f77-a991-a34e3961e213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:58.539067 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d2434-1100-4f77-a991-a34e3961e213 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:58.539339 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d2434-1100-4f77-a991-a34e3961e213 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:58.539651 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d2434-1100-4f77-a991-a34e3961e213 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:58.611358 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9d2434-1100-4f77-a991-a34e3961e213 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:58.612854 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9d2434-1100-4f77-a991-a34e3961e213 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:58.705673 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:58.713256 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 164/316] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:47:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:47:58.724183 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759c433-d960-403e-a933-7c35079c912b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:58.831107 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759c433-d960-403e-a933-7c35079c912b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:58.866904 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c754d-50d2-444e-9b89-ce59e6386d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:58.867542 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c754d-50d2-444e-9b89-ce59e6386d66 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:58.867827 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c754d-50d2-444e-9b89-ce59e6386d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:58.868105 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c754d-50d2-444e-9b89-ce59e6386d66 None admin] PATH_INFO: `/v3/projects/5b12c253c6eb48c9a549a9a8436eced1` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:58.869269 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3c754d-50d2-444e-9b89-ce59e6386d66 None admin] RBAC: Authorizing `identity:get_project(project_id=5b12c253c6eb48c9a549a9a8436eced1)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:58.870706 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3c754d-50d2-444e-9b89-ce59e6386d66 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:58.872521 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 153/317] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:47:58 2022] GET /identity/v3/projects/5b12c253c6eb48c9a549a9a8436eced1 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:58.878579 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9d2434-1100-4f77-a991-a34e3961e213 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:58.990893 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9d2434-1100-4f77-a991-a34e3961e213 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:59.029259 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48118dee-3419-4430-bd56-c1d57dae54ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:59.030233 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48118dee-3419-4430-bd56-c1d57dae54ab None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:59.030757 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48118dee-3419-4430-bd56-c1d57dae54ab None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:59.031302 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48118dee-3419-4430-bd56-c1d57dae54ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:59.032261 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48118dee-3419-4430-bd56-c1d57dae54ab None admin] RBAC: Adding query filter params (role.id=8f7a2ac695c6482cbf48819cee250c22, scope.project.id=5b12c253c6eb48c9a549a9a8436eced1, user.id=d11f31e750454372b47b35c41fcc9726) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:47:59.032890 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48118dee-3419-4430-bd56-c1d57dae54ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:59.034527 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48118dee-3419-4430-bd56-c1d57dae54ab None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:59.042469 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 165/318] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:47:58 2022] GET /identity/v3/role_assignments?user.id=d11f31e750454372b47b35c41fcc9726&scope.project.id=5b12c253c6eb48c9a549a9a8436eced1&role.id=8f7a2ac695c6482cbf48819cee250c22 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:47:59.117737 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a26d4-820c-4c91-9df8-dc7f065c515d None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:59.118224 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a26d4-820c-4c91-9df8-dc7f065c515d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:59.118224 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a26d4-820c-4c91-9df8-dc7f065c515d None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:59.119862 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 154/319] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:59.125591 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1b77e-1639-4b9e-9af3-3e3600623c09 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:59.125906 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1b77e-1639-4b9e-9af3-3e3600623c09 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:59.126148 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1b77e-1639-4b9e-9af3-3e3600623c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:59.148573 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-d2e1b77e-1639-4b9e-9af3-3e3600623c09 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:59.149819 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d2e1b77e-1639-4b9e-9af3-3e3600623c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:59.208059 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:59.228872 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 166/320] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:59.378586 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b75e5a-dcf7-4206-ad82-3859d0ebdfbd None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:59.379388 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b75e5a-dcf7-4206-ad82-3859d0ebdfbd None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:59.379746 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b75e5a-dcf7-4206-ad82-3859d0ebdfbd None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:59.381260 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 155/321] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:59.386125 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761acbc-8020-428c-b160-eed20be44add None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:59.386419 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761acbc-8020-428c-b160-eed20be44add None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:59.386693 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761acbc-8020-428c-b160-eed20be44add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:59.405191 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-0761acbc-8020-428c-b160-eed20be44add None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:59.406367 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-0761acbc-8020-428c-b160-eed20be44add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:59.456116 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7bdfda-c2e7-49e2-bbd6-5306a4fa461b None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:59.456562 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7bdfda-c2e7-49e2-bbd6-5306a4fa461b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:59.457001 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7bdfda-c2e7-49e2-bbd6-5306a4fa461b None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:59.459072 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 156/322] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:47:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:47:59.463955 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888cf85-498a-463e-ba1e-30e7d9b1d4c3 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:59.464248 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888cf85-498a-463e-ba1e-30e7d9b1d4c3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:59.464560 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888cf85-498a-463e-ba1e-30e7d9b1d4c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:59.477439 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 167/323] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:47:59.477608 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:59.483073 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-b888cf85-498a-463e-ba1e-30e7d9b1d4c3 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:47:59.484263 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b888cf85-498a-463e-ba1e-30e7d9b1d4c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:47:59.505615 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761acbc-8020-428c-b160-eed20be44add None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:59.555185 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 157/324] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:47:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 00:47:59.609977 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761acbc-8020-428c-b160-eed20be44add None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:59.624062 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:59.644474 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb9314-d630-4c07-94be-620aacd75402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:59.645534 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb9314-d630-4c07-94be-620aacd75402 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:59.645842 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb9314-d630-4c07-94be-620aacd75402 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:59.646122 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb9314-d630-4c07-94be-620aacd75402 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:59.647229 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bb9314-d630-4c07-94be-620aacd75402 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:59.648805 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bb9314-d630-4c07-94be-620aacd75402 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:59.650571 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 168/325] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:47:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:47:59.655926 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b888cf85-498a-463e-ba1e-30e7d9b1d4c3 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:59.743492 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb9314-d630-4c07-94be-620aacd75402 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:59.756711 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b888cf85-498a-463e-ba1e-30e7d9b1d4c3 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:59.795167 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d3ba3-8c0b-4f59-8c32-88542de06a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:59.795836 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d3ba3-8c0b-4f59-8c32-88542de06a6b None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:59.796122 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d3ba3-8c0b-4f59-8c32-88542de06a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:59.796400 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d3ba3-8c0b-4f59-8c32-88542de06a6b None admin] PATH_INFO: `/v3/groups` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:59.797104 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d3ba3-8c0b-4f59-8c32-88542de06a6b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:47:59.798543 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d3ba3-8c0b-4f59-8c32-88542de06a6b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:47:59.806436 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:47:59.845418 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb9314-d630-4c07-94be-620aacd75402 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:47:59.878109 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f5ed8-34bf-4870-866f-d4b2fac61630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:47:59.879341 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f5ed8-34bf-4870-866f-d4b2fac61630 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:47:59.879626 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f5ed8-34bf-4870-866f-d4b2fac61630 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:47:59.879885 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f5ed8-34bf-4870-866f-d4b2fac61630 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:47:59.889615 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f5ed8-34bf-4870-866f-d4b2fac61630 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:47:59.894383 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-db5f5ed8-34bf-4870-866f-d4b2fac61630 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 00:47:59.894383 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:47:59.894383 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:47:59.894383 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:47:59.894383 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:47:59.894383 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:47:59.894383 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:47:59.894383 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:47:59.894383 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:47:59.894383 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:47:59.894383 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:47:59.894383 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:47:59.894383 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:47:59.894383 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:47:59.894383 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:47:59.894383 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:47:59.895465 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:47:59.895465 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:47:59.895465 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:47:59.895465 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:47:59.895465 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:47:59.895465 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:47:59.895465 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:47:59.895465 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:47:59.895465 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:47:59.895465 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:47:59.895465 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:47:59.895465 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:47:59.895465 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:47:59.895465 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:47:59.895465 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:47:59.895465 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:47:59.896278 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:47:59.896278 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:47:59.896278 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:47:59.896278 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:47:59.896278 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:47:59.896278 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:47:59.896278 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:47:59.896278 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:47:59.896278 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:47:59.896278 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:47:59.896278 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 00:47:59.896278 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:47:59.900443 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 169/326] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:47:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:47:59.901222 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f5ed8-34bf-4870-866f-d4b2fac61630 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:59.901845 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 158/327] 10.222.0.27 () {60 vars in 1134 bytes} [Sun Sep 4 00:47:59 2022] POST /identity/v3/groups => generated 226 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:47:59.902669 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d3ba3-8c0b-4f59-8c32-88542de06a6b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:47:59.955602 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:00.020259 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f5ed8-34bf-4870-866f-d4b2fac61630 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:00.022644 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d3ba3-8c0b-4f59-8c32-88542de06a6b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:00.060110 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91343cf5-e29d-4368-b99c-5858898859f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:00.061372 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91343cf5-e29d-4368-b99c-5858898859f8 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:00.061372 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91343cf5-e29d-4368-b99c-5858898859f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:00.061372 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91343cf5-e29d-4368-b99c-5858898859f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:00.061912 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91343cf5-e29d-4368-b99c-5858898859f8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:00.062213 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91343cf5-e29d-4368-b99c-5858898859f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:00.063315 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d465ad-d530-47d9-a016-a8e8e1ddfbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:00.063587 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91343cf5-e29d-4368-b99c-5858898859f8 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:00.063955 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d465ad-d530-47d9-a016-a8e8e1ddfbc3 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:00.064242 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d465ad-d530-47d9-a016-a8e8e1ddfbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:00.064597 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d465ad-d530-47d9-a016-a8e8e1ddfbc3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:00.073325 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 159/328] 10.222.0.27 () {56 vars in 1101 bytes} [Sun Sep 4 00:47:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:00.076548 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d465ad-d530-47d9-a016-a8e8e1ddfbc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:00.079520 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91343cf5-e29d-4368-b99c-5858898859f8 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:00.081812 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-55d465ad-d530-47d9-a016-a8e8e1ddfbc3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:48:00.081812 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:00.081812 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:00.081812 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:00.081812 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:00.081812 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:00.081812 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:00.081812 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:00.081812 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:00.081812 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:00.081812 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:00.081812 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:00.081812 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:00.081812 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:00.081812 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:00.081812 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:00.082838 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:00.082838 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:00.082838 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:00.082838 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:00.082838 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:00.082838 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:00.082838 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:00.082838 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:00.082838 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:00.082838 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:00.082838 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:00.082838 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:00.082838 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:00.082838 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:00.082838 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:00.082838 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:00.083607 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:00.083607 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:00.083607 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:00.083607 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:00.083607 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:00.083607 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:00.083607 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:00.083607 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:00.083607 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:00.083607 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:00.083607 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:48:00.083607 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:00.083607 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 170/329] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:47:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:00.088887 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d465ad-d530-47d9-a016-a8e8e1ddfbc3 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:00.151283 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:00.210319 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91343cf5-e29d-4368-b99c-5858898859f8 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:00.217367 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d465ad-d530-47d9-a016-a8e8e1ddfbc3 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:00.219307 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:00.261127 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da81a74-6e6e-40d7-a956-9db5223ef007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:00.262135 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da81a74-6e6e-40d7-a956-9db5223ef007 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:00.262135 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da81a74-6e6e-40d7-a956-9db5223ef007 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:00.262325 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da81a74-6e6e-40d7-a956-9db5223ef007 None admin] PATH_INFO: `/v3/users/0ce42cd5938040f1a4f5925b0238dd68` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:00.263500 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da81a74-6e6e-40d7-a956-9db5223ef007 None admin] RBAC: Authorizing `identity:get_user(user_id=0ce42cd5938040f1a4f5925b0238dd68)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:00.265229 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da81a74-6e6e-40d7-a956-9db5223ef007 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:00.268999 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 160/330] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:48:00 2022] GET /identity/v3/users/0ce42cd5938040f1a4f5925b0238dd68 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:00.272252 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d3ed31-eca8-40c7-9cc8-1878d660a8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:00.272900 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d3ed31-eca8-40c7-9cc8-1878d660a8d8 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:00.273179 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d3ed31-eca8-40c7-9cc8-1878d660a8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:00.273440 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da81a74-6e6e-40d7-a956-9db5223ef007 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:00.273539 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d3ed31-eca8-40c7-9cc8-1878d660a8d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:00.274107 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d3ed31-eca8-40c7-9cc8-1878d660a8d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:00.274407 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d3ed31-eca8-40c7-9cc8-1878d660a8d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:00.275775 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d3ed31-eca8-40c7-9cc8-1878d660a8d8 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:00.288134 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 171/331] 10.222.0.27 () {56 vars in 1101 bytes} [Sun Sep 4 00:48:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:48:00.294207 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d3ed31-eca8-40c7-9cc8-1878d660a8d8 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:00.401585 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da81a74-6e6e-40d7-a956-9db5223ef007 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:00.416874 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d3ed31-eca8-40c7-9cc8-1878d660a8d8 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:00.438317 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:00.443583 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af27e948-5af0-41aa-8ed5-255df76f29cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:00.444515 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27e948-5af0-41aa-8ed5-255df76f29cc None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:00.445096 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27e948-5af0-41aa-8ed5-255df76f29cc None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:00.445599 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27e948-5af0-41aa-8ed5-255df76f29cc None admin] PATH_INFO: `/v3/system/users/0ce42cd5938040f1a4f5925b0238dd68/roles/8f7a2ac695c6482cbf48819cee250c22` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:00.447409 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af27e948-5af0-41aa-8ed5-255df76f29cc None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0ce42cd5938040f1a4f5925b0238dd68, role_id=8f7a2ac695c6482cbf48819cee250c22)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:00.449334 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af27e948-5af0-41aa-8ed5-255df76f29cc None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:00.453905 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8956f-81da-4825-a021-2cfe11251cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:00.454663 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8956f-81da-4825-a021-2cfe11251cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:00.455016 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8956f-81da-4825-a021-2cfe11251cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:00.455285 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8956f-81da-4825-a021-2cfe11251cf7 None admin] PATH_INFO: `/v3/users/b978689a22cc407ea316eba38b779314` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:00.457945 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b8956f-81da-4825-a021-2cfe11251cf7 None admin] RBAC: Authorizing `identity:get_user(user_id=b978689a22cc407ea316eba38b779314)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:00.458684 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b8956f-81da-4825-a021-2cfe11251cf7 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:00.464490 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 172/332] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:48:00 2022] GET /identity/v3/users/b978689a22cc407ea316eba38b779314 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:00.466342 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8956f-81da-4825-a021-2cfe11251cf7 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:00.502180 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:00.532491 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 161/333] 10.222.0.27 () {58 vars in 1334 bytes} [Sun Sep 4 00:48:00 2022] PUT /identity/v3/system/users/0ce42cd5938040f1a4f5925b0238dd68/roles/8f7a2ac695c6482cbf48819cee250c22 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 00:48:00.580688 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8956f-81da-4825-a021-2cfe11251cf7 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:00.622581 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b785651-61b3-4e9e-b0dc-9097c03048c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:00.623203 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b785651-61b3-4e9e-b0dc-9097c03048c6 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:00.623481 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b785651-61b3-4e9e-b0dc-9097c03048c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:00.623753 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b785651-61b3-4e9e-b0dc-9097c03048c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:00.624502 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b785651-61b3-4e9e-b0dc-9097c03048c6 None admin] RBAC: Adding query filter params (role.id=d1261c99e1d64f9da63adfff739561e8, user.id=b978689a22cc407ea316eba38b779314, scope.system=all) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:00.624814 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b785651-61b3-4e9e-b0dc-9097c03048c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:00.626296 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b785651-61b3-4e9e-b0dc-9097c03048c6 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:00.641890 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 173/334] 10.222.0.27 () {56 vars in 1308 bytes} [Sun Sep 4 00:48:00 2022] GET /identity/v3/role_assignments?user.id=b978689a22cc407ea316eba38b779314&scope.system=all&role.id=d1261c99e1d64f9da63adfff739561e8 => generated 510 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:02.403482 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c290414-9d94-4898-bf47-7abcfdecbac4 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:02.403974 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c290414-9d94-4898-bf47-7abcfdecbac4 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:02.404080 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c290414-9d94-4898-bf47-7abcfdecbac4 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:02.405402 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 162/335] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:02.419627 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c628779-f4c5-4c6d-b559-562c95d26d2f None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:02.420086 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c628779-f4c5-4c6d-b559-562c95d26d2f None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:02.420159 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c628779-f4c5-4c6d-b559-562c95d26d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:02.445448 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-9c628779-f4c5-4c6d-b559-562c95d26d2f None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:02.446653 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-9c628779-f4c5-4c6d-b559-562c95d26d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:02.534130 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 174/336] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:02.547251 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5dc8ba-fb91-45b5-a2ca-651637e66d02 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:02.547529 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5dc8ba-fb91-45b5-a2ca-651637e66d02 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:02.547808 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5dc8ba-fb91-45b5-a2ca-651637e66d02 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:02.554805 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 163/337] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:02.569782 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5e4a6-eac3-4417-8a36-53700cf1d8db None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:02.570058 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5e4a6-eac3-4417-8a36-53700cf1d8db None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:02.570332 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5e4a6-eac3-4417-8a36-53700cf1d8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:02.596937 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-b1d5e4a6-eac3-4417-8a36-53700cf1d8db None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:02.598275 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b1d5e4a6-eac3-4417-8a36-53700cf1d8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:02.618292 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:02.681508 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 175/338] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:02.770337 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048dbf0-3020-4649-8a3d-48ceee1c2695 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:02.770659 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048dbf0-3020-4649-8a3d-48ceee1c2695 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:02.771082 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048dbf0-3020-4649-8a3d-48ceee1c2695 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:02.772738 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 164/339] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:02.778384 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71264911-2cb8-4792-923a-3bc0eeabc7b7 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:02.778671 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71264911-2cb8-4792-923a-3bc0eeabc7b7 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:02.778952 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71264911-2cb8-4792-923a-3bc0eeabc7b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:02.805527 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-71264911-2cb8-4792-923a-3bc0eeabc7b7 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:02.806642 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-71264911-2cb8-4792-923a-3bc0eeabc7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:02.886628 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 176/340] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:02.944695 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b048dbf0-3020-4649-8a3d-48ceee1c2695 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:03.065561 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b048dbf0-3020-4649-8a3d-48ceee1c2695 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:03.111819 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21eef9-0412-4ce5-b978-7fef81c97d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:03.112499 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21eef9-0412-4ce5-b978-7fef81c97d1d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:03.112747 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21eef9-0412-4ce5-b978-7fef81c97d1d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:03.113615 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21eef9-0412-4ce5-b978-7fef81c97d1d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:03.129947 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de21eef9-0412-4ce5-b978-7fef81c97d1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:03.134856 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-de21eef9-0412-4ce5-b978-7fef81c97d1d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 00:48:03.134856 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:03.134856 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:03.134856 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:03.134856 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:03.134856 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:03.134856 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:03.134856 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:03.134856 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:03.134856 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:03.134856 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:03.134856 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:03.134856 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:03.134856 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:03.134856 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:03.134856 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:03.135687 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:03.135687 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:03.135687 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:03.135687 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:03.135687 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:03.135687 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:03.135687 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:03.135687 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:03.135687 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:03.135687 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:03.135687 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:03.135687 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:03.135687 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:03.135687 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:03.135687 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:03.135687 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:03.136474 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:03.136474 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:03.136474 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:03.136474 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:03.136474 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:03.136474 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:03.136474 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:03.136474 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:03.136474 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:03.136474 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:03.136474 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 00:48:03.136474 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:03.136474 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 165/341] 10.222.0.27 () {56 vars in 1114 bytes} [Sun Sep 4 00:48:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:03.143935 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71264911-2cb8-4792-923a-3bc0eeabc7b7 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:03.268294 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:03.290680 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71264911-2cb8-4792-923a-3bc0eeabc7b7 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:03.312748 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21eef9-0412-4ce5-b978-7fef81c97d1d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:03.346161 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d0912b-d004-49f3-8fa4-8e60bf48728b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:03.346702 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0912b-d004-49f3-8fa4-8e60bf48728b None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:03.346987 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0912b-d004-49f3-8fa4-8e60bf48728b None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:03.347266 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0912b-d004-49f3-8fa4-8e60bf48728b None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:03.348872 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d0912b-d004-49f3-8fa4-8e60bf48728b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:03.349207 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d0912b-d004-49f3-8fa4-8e60bf48728b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:03.350608 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d0912b-d004-49f3-8fa4-8e60bf48728b None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:03.367890 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 177/342] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:48:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:03.373472 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d0912b-d004-49f3-8fa4-8e60bf48728b None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:03.412769 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:03.520709 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21eef9-0412-4ce5-b978-7fef81c97d1d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:03.529703 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d0912b-d004-49f3-8fa4-8e60bf48728b None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:03.570680 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63597d0b-c312-4fad-8680-1cb5f14f3869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:03.571703 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63597d0b-c312-4fad-8680-1cb5f14f3869 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:03.572503 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63597d0b-c312-4fad-8680-1cb5f14f3869 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:03.573186 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63597d0b-c312-4fad-8680-1cb5f14f3869 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:03.577610 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b687007-f893-47b9-89a3-5665a9aaece6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:03.578339 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b687007-f893-47b9-89a3-5665a9aaece6 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:03.578739 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b687007-f893-47b9-89a3-5665a9aaece6 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:03.579115 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b687007-f893-47b9-89a3-5665a9aaece6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:03.588989 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63597d0b-c312-4fad-8680-1cb5f14f3869 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:03.595426 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-63597d0b-c312-4fad-8680-1cb5f14f3869 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:48:03.595426 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:03.595426 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:03.595426 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:03.595426 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:03.595426 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:03.595426 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:03.595426 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:03.595426 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:03.595426 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:03.595426 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:03.595426 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:03.595426 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:03.595426 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:03.595426 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:03.595426 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:03.596369 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:03.596369 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:03.596369 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:03.596369 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:03.596369 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:03.596369 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:03.596369 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:03.596369 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:03.596369 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:03.596369 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:03.596369 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:03.596369 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:03.596369 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:03.596369 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:03.596369 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:03.596369 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:03.598464 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:03.598464 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:03.598464 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:03.598464 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:03.598464 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:03.598464 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:03.598464 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:03.598464 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:03.598464 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:03.598464 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:03.598464 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:48:03.598464 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:03.599348 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:03.601578 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 166/343] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:48:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:03.603454 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabcc8d-ace2-4027-8f39-44f7c1cb7129 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:03.610358 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabcc8d-ace2-4027-8f39-44f7c1cb7129 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:03.610358 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabcc8d-ace2-4027-8f39-44f7c1cb7129 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:03.610358 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 167/344] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:03.610358 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eabcc8d-ace2-4027-8f39-44f7c1cb7129 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:03.670963 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b687007-f893-47b9-89a3-5665a9aaece6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:03.672719 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b687007-f893-47b9-89a3-5665a9aaece6 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:03.740399 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eabcc8d-ace2-4027-8f39-44f7c1cb7129 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:03.754056 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:03.780915 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d3c658-5b74-4cda-901e-ee29e9573ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:03.781560 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d3c658-5b74-4cda-901e-ee29e9573ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:03.782131 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d3c658-5b74-4cda-901e-ee29e9573ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:03.782464 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d3c658-5b74-4cda-901e-ee29e9573ed5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:03.788315 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 178/345] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:48:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:48:03.789205 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b687007-f893-47b9-89a3-5665a9aaece6 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:03.796418 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d3c658-5b74-4cda-901e-ee29e9573ed5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:03.803999 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-54d3c658-5b74-4cda-901e-ee29e9573ed5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:48:03.803999 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:03.803999 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:03.803999 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:03.803999 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:03.803999 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:03.803999 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:03.803999 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:03.803999 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:03.803999 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:03.803999 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:03.803999 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:03.803999 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:03.803999 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:03.803999 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:03.803999 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:03.804962 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:03.804962 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:03.804962 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:03.804962 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:03.804962 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:03.804962 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:03.804962 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:03.804962 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:03.804962 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:03.804962 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:03.804962 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:03.804962 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:03.804962 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:03.804962 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:03.804962 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:03.804962 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:03.806266 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:03.806266 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:03.806266 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:03.806266 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:03.806266 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:03.806266 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:03.806266 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:03.806266 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:03.806266 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:03.806266 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:03.806266 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:48:03.806266 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:03.806266 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 168/346] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:48:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:03.808063 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43e875-e7c0-4797-98b1-3277bc0101dd None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:03.808063 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43e875-e7c0-4797-98b1-3277bc0101dd None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:03.811826 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43e875-e7c0-4797-98b1-3277bc0101dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:03.834238 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-fa43e875-e7c0-4797-98b1-3277bc0101dd None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:03.835463 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-fa43e875-e7c0-4797-98b1-3277bc0101dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:03.842645 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:03.924315 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b687007-f893-47b9-89a3-5665a9aaece6 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:03.928593 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 169/347] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:03.929861 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43e875-e7c0-4797-98b1-3277bc0101dd None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:03.965163 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40149afe-b7fc-41ac-b2d0-2384107880bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:03.965819 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40149afe-b7fc-41ac-b2d0-2384107880bb None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:03.966100 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40149afe-b7fc-41ac-b2d0-2384107880bb None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:03.966374 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40149afe-b7fc-41ac-b2d0-2384107880bb None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:03.967144 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40149afe-b7fc-41ac-b2d0-2384107880bb None admin] RBAC: Adding query filter params (name=member) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:03.967452 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40149afe-b7fc-41ac-b2d0-2384107880bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:03.969003 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40149afe-b7fc-41ac-b2d0-2384107880bb None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:03.979230 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 179/348] 10.222.0.27 () {56 vars in 1101 bytes} [Sun Sep 4 00:48:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:03.979864 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40149afe-b7fc-41ac-b2d0-2384107880bb None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:04.046004 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:04.049551 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43e875-e7c0-4797-98b1-3277bc0101dd None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:04.089409 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b614cf-2626-4d7d-aeed-afea6db9fcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:04.090050 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b614cf-2626-4d7d-aeed-afea6db9fcbe None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:04.090331 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b614cf-2626-4d7d-aeed-afea6db9fcbe None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:04.090605 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b614cf-2626-4d7d-aeed-afea6db9fcbe None admin] PATH_INFO: `/v3/projects/c8fe5139a2a24ec780814c7f39d5b437` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:04.091786 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b614cf-2626-4d7d-aeed-afea6db9fcbe None admin] RBAC: Authorizing `identity:get_project(project_id=c8fe5139a2a24ec780814c7f39d5b437)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:04.092134 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40149afe-b7fc-41ac-b2d0-2384107880bb None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:04.093316 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b614cf-2626-4d7d-aeed-afea6db9fcbe None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:04.095785 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:04.100911 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 170/349] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:48:03 2022] GET /identity/v3/projects/c8fe5139a2a24ec780814c7f39d5b437 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:04.100911 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26950fb-605b-424e-acb7-177f5ebbc487 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:04.100911 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26950fb-605b-424e-acb7-177f5ebbc487 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:04.100911 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26950fb-605b-424e-acb7-177f5ebbc487 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:04.100911 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 171/350] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:04.101640 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26950fb-605b-424e-acb7-177f5ebbc487 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:04.142088 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456774f8-652c-42da-8803-64e44948b96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:04.143501 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456774f8-652c-42da-8803-64e44948b96c None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:04.144163 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456774f8-652c-42da-8803-64e44948b96c None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:04.144719 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456774f8-652c-42da-8803-64e44948b96c None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:04.145671 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456774f8-652c-42da-8803-64e44948b96c None admin] RBAC: Adding query filter params (name=member) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:04.146124 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456774f8-652c-42da-8803-64e44948b96c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:04.147697 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456774f8-652c-42da-8803-64e44948b96c None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:04.163258 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 180/351] 10.222.0.27 () {56 vars in 1101 bytes} [Sun Sep 4 00:48:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:04.164974 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19664ed-d91f-4c18-8c60-dacc5eadf881 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:04.165427 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19664ed-d91f-4c18-8c60-dacc5eadf881 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:04.166011 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19664ed-d91f-4c18-8c60-dacc5eadf881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:04.188928 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-b19664ed-d91f-4c18-8c60-dacc5eadf881 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:04.190165 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b19664ed-d91f-4c18-8c60-dacc5eadf881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:04.238290 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26950fb-605b-424e-acb7-177f5ebbc487 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:04.270183 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 181/352] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:04.271286 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19664ed-d91f-4c18-8c60-dacc5eadf881 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:04.293081 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f729a-2e6f-4d89-b483-589f8037f9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:04.293842 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f729a-2e6f-4d89-b483-589f8037f9d1 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:04.294772 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f729a-2e6f-4d89-b483-589f8037f9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:04.295183 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f729a-2e6f-4d89-b483-589f8037f9d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:04.335260 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:04.372579 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648f729a-2e6f-4d89-b483-589f8037f9d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:04.374386 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648f729a-2e6f-4d89-b483-589f8037f9d1 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:04.375301 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:04.381776 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19664ed-d91f-4c18-8c60-dacc5eadf881 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:04.417881 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1966128-2527-4633-983b-7b078c0bbe5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:04.418781 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1966128-2527-4633-983b-7b078c0bbe5e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:04.419289 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1966128-2527-4633-983b-7b078c0bbe5e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:04.419733 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1966128-2527-4633-983b-7b078c0bbe5e None admin] PATH_INFO: `/v3/groups/664a63a166be4fd9a4e70be79f046c5f` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:04.426993 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1966128-2527-4633-983b-7b078c0bbe5e None admin] RBAC: Authorizing `identity:get_group(group_id=664a63a166be4fd9a4e70be79f046c5f)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:04.428713 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1966128-2527-4633-983b-7b078c0bbe5e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:04.430728 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 182/353] 10.222.0.27 () {56 vars in 1180 bytes} [Sun Sep 4 00:48:04 2022] GET /identity/v3/groups/664a63a166be4fd9a4e70be79f046c5f => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:04.431805 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1966128-2527-4633-983b-7b078c0bbe5e None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:04.482869 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 172/354] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:48:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:48:04.483637 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f729a-2e6f-4d89-b483-589f8037f9d1 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:04.542681 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1966128-2527-4633-983b-7b078c0bbe5e None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:04.558574 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:04.580743 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0491ba2b-120f-4f4b-a4e1-4d9e5cd93acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:04.582256 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491ba2b-120f-4f4b-a4e1-4d9e5cd93acc None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:04.582734 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491ba2b-120f-4f4b-a4e1-4d9e5cd93acc None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:04.583220 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491ba2b-120f-4f4b-a4e1-4d9e5cd93acc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:04.599508 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f729a-2e6f-4d89-b483-589f8037f9d1 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:04.631613 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:04.650509 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1133910-5a9d-4efa-b113-531b2e7ea6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:04.651149 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1133910-5a9d-4efa-b113-531b2e7ea6b3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:04.651598 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1133910-5a9d-4efa-b113-531b2e7ea6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:04.651871 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1133910-5a9d-4efa-b113-531b2e7ea6b3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:04.668431 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1133910-5a9d-4efa-b113-531b2e7ea6b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:04.674295 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-a1133910-5a9d-4efa-b113-531b2e7ea6b3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 00:48:04.674295 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:04.674295 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:04.674295 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:04.674295 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:04.674295 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:04.674295 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:04.674295 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:04.674295 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:04.674295 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:04.674295 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:04.674295 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:04.674295 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:04.674295 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:04.674295 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:04.674295 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:04.675194 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:04.675194 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:04.675194 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:04.675194 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:04.675194 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:04.675194 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:04.675194 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:04.675194 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:04.675194 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:04.675194 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:04.675194 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:04.675194 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:04.675194 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:04.675194 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:04.675194 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:04.675194 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:04.676860 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:04.676860 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:04.676860 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:04.676860 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:04.676860 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:04.676860 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:04.676860 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:04.676860 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:04.676860 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:04.676860 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:04.676860 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 04 00:48:04.676860 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:04.680917 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 173/355] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:48:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:04.680917 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1133910-5a9d-4efa-b113-531b2e7ea6b3 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:04.690119 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0491ba2b-120f-4f4b-a4e1-4d9e5cd93acc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:04.691849 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0491ba2b-120f-4f4b-a4e1-4d9e5cd93acc None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:04.812596 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1133910-5a9d-4efa-b113-531b2e7ea6b3 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:04.830577 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 183/356] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:48:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:48:04.831433 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0491ba2b-120f-4f4b-a4e1-4d9e5cd93acc None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:04.850777 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:04.865487 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f85e59-8ab2-47d2-80ae-4b0b59c7e147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:04.866106 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f85e59-8ab2-47d2-80ae-4b0b59c7e147 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:04.866391 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f85e59-8ab2-47d2-80ae-4b0b59c7e147 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:04.866668 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f85e59-8ab2-47d2-80ae-4b0b59c7e147 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:04.867429 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f85e59-8ab2-47d2-80ae-4b0b59c7e147 None admin] RBAC: Adding query filter params (group.id=664a63a166be4fd9a4e70be79f046c5f, role.id=7df9ebc56a2a4214b2b64a5941812ef0, scope.project.id=c8fe5139a2a24ec780814c7f39d5b437) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:04.867742 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f85e59-8ab2-47d2-80ae-4b0b59c7e147 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:04.869259 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f85e59-8ab2-47d2-80ae-4b0b59c7e147 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:04.875195 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 174/357] 10.222.0.27 () {56 vars in 1376 bytes} [Sun Sep 4 00:48:04 2022] GET /identity/v3/role_assignments?group.id=664a63a166be4fd9a4e70be79f046c5f&scope.project.id=c8fe5139a2a24ec780814c7f39d5b437&role.id=7df9ebc56a2a4214b2b64a5941812ef0 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:04.875852 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f85e59-8ab2-47d2-80ae-4b0b59c7e147 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:04.901515 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:04.977501 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0491ba2b-120f-4f4b-a4e1-4d9e5cd93acc None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:05.022415 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd986cd4-61da-4513-8ae6-4fee5fc4dc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:05.022415 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd986cd4-61da-4513-8ae6-4fee5fc4dc17 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:05.022415 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd986cd4-61da-4513-8ae6-4fee5fc4dc17 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:05.022415 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd986cd4-61da-4513-8ae6-4fee5fc4dc17 None admin] PATH_INFO: `/v3/projects/c8fe5139a2a24ec780814c7f39d5b437` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:05.022862 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f85e59-8ab2-47d2-80ae-4b0b59c7e147 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:05.023152 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd986cd4-61da-4513-8ae6-4fee5fc4dc17 None admin] RBAC: Authorizing `identity:get_project(project_id=c8fe5139a2a24ec780814c7f39d5b437)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:05.025397 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd986cd4-61da-4513-8ae6-4fee5fc4dc17 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:05.027142 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 184/358] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:48:04 2022] GET /identity/v3/projects/c8fe5139a2a24ec780814c7f39d5b437 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:05.027918 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd986cd4-61da-4513-8ae6-4fee5fc4dc17 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:05.061394 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be748bd0-58ac-4e8d-b7da-e4d440b963ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:05.062091 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be748bd0-58ac-4e8d-b7da-e4d440b963ae None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:05.062303 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be748bd0-58ac-4e8d-b7da-e4d440b963ae None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:05.062581 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be748bd0-58ac-4e8d-b7da-e4d440b963ae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:05.074918 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be748bd0-58ac-4e8d-b7da-e4d440b963ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:05.080191 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-be748bd0-58ac-4e8d-b7da-e4d440b963ae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 00:48:05.080191 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:05.080191 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:05.080191 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:05.080191 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:05.080191 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:05.080191 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:05.080191 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:05.080191 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:05.080191 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:05.080191 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:05.080191 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:05.080191 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:05.080191 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:05.080191 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:05.080191 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:05.082472 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:05.082472 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:05.082472 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:05.082472 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:05.082472 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:05.082472 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:05.082472 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:05.082472 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:05.082472 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:05.082472 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:05.082472 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:05.082472 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:05.082472 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:05.082472 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:05.082472 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:05.082472 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:05.083380 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:05.083380 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:05.083380 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:05.083380 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:05.083380 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:05.083380 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:05.083380 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:05.083380 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:05.083380 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:05.083380 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:05.083380 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 00:48:05.083380 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:05.083380 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 175/359] 10.222.0.27 () {56 vars in 1114 bytes} [Sun Sep 4 00:48:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:05.083380 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be748bd0-58ac-4e8d-b7da-e4d440b963ae None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:05.153966 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:05.167978 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd986cd4-61da-4513-8ae6-4fee5fc4dc17 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:05.203100 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:05.215158 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c1842-cdc8-4af5-9956-94b02f35dddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:05.216017 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c1842-cdc8-4af5-9956-94b02f35dddb None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:05.216446 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c1842-cdc8-4af5-9956-94b02f35dddb None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:05.216773 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c1842-cdc8-4af5-9956-94b02f35dddb None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:05.217447 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46c1842-cdc8-4af5-9956-94b02f35dddb None admin] RBAC: Adding query filter params (name=reader) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:05.217752 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46c1842-cdc8-4af5-9956-94b02f35dddb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:05.218861 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be748bd0-58ac-4e8d-b7da-e4d440b963ae None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:05.219151 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46c1842-cdc8-4af5-9956-94b02f35dddb None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:05.236624 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 185/360] 10.222.0.27 () {56 vars in 1101 bytes} [Sun Sep 4 00:48:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:05.238278 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc51cae-6a41-4870-bff5-a552559b819d None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:05.238540 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc51cae-6a41-4870-bff5-a552559b819d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:05.238928 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc51cae-6a41-4870-bff5-a552559b819d None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:05.240136 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 186/361] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:05.240897 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc51cae-6a41-4870-bff5-a552559b819d None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:05.274457 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ed73f-45a7-49a2-a482-451e309f7dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:05.275438 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ed73f-45a7-49a2-a482-451e309f7dd7 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:05.275733 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ed73f-45a7-49a2-a482-451e309f7dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:05.276115 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ed73f-45a7-49a2-a482-451e309f7dd7 None admin] PATH_INFO: `/v3/projects/5b12c253c6eb48c9a549a9a8436eced1` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:05.277731 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5ed73f-45a7-49a2-a482-451e309f7dd7 None admin] RBAC: Authorizing `identity:get_project(project_id=5b12c253c6eb48c9a549a9a8436eced1)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:05.279590 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5ed73f-45a7-49a2-a482-451e309f7dd7 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:05.281976 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 176/362] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:48:05 2022] GET /identity/v3/projects/5b12c253c6eb48c9a549a9a8436eced1 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:05.282783 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ed73f-45a7-49a2-a482-451e309f7dd7 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:05.375804 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc51cae-6a41-4870-bff5-a552559b819d None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:05.407097 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ed73f-45a7-49a2-a482-451e309f7dd7 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:05.417448 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5808439-c438-4e1d-b721-333fe2190cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:05.417448 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5808439-c438-4e1d-b721-333fe2190cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:05.417448 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5808439-c438-4e1d-b721-333fe2190cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:05.417973 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5808439-c438-4e1d-b721-333fe2190cf5 None admin] PATH_INFO: `/v3/groups/664a63a166be4fd9a4e70be79f046c5f` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:05.419696 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5808439-c438-4e1d-b721-333fe2190cf5 None admin] RBAC: Authorizing `identity:get_group(group_id=664a63a166be4fd9a4e70be79f046c5f)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:05.421578 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5808439-c438-4e1d-b721-333fe2190cf5 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:05.424613 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 187/363] 10.222.0.27 () {56 vars in 1180 bytes} [Sun Sep 4 00:48:05 2022] GET /identity/v3/groups/664a63a166be4fd9a4e70be79f046c5f => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:05.426073 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832fe06b-a99e-486a-847b-ad489c5ed696 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:05.426353 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832fe06b-a99e-486a-847b-ad489c5ed696 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:05.427051 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832fe06b-a99e-486a-847b-ad489c5ed696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:05.433203 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:05.450482 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee07283f-b8ae-4778-b3bd-7e2731b29d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:05.450988 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-832fe06b-a99e-486a-847b-ad489c5ed696 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:05.451107 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07283f-b8ae-4778-b3bd-7e2731b29d78 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:05.451358 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07283f-b8ae-4778-b3bd-7e2731b29d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:05.451632 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07283f-b8ae-4778-b3bd-7e2731b29d78 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:05.452971 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-832fe06b-a99e-486a-847b-ad489c5ed696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:05.453062 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee07283f-b8ae-4778-b3bd-7e2731b29d78 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:05.453062 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee07283f-b8ae-4778-b3bd-7e2731b29d78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:05.454164 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee07283f-b8ae-4778-b3bd-7e2731b29d78 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:05.465270 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 177/364] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:48:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:05.466183 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee07283f-b8ae-4778-b3bd-7e2731b29d78 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:05.518443 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:05.534501 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 188/365] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:05.535578 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832fe06b-a99e-486a-847b-ad489c5ed696 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:05.599227 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee07283f-b8ae-4778-b3bd-7e2731b29d78 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:05.642829 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1e8cd-0f76-4870-8f8b-db7b44fb9062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:05.643413 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1e8cd-0f76-4870-8f8b-db7b44fb9062 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:05.643698 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1e8cd-0f76-4870-8f8b-db7b44fb9062 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:05.643978 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1e8cd-0f76-4870-8f8b-db7b44fb9062 None admin] PATH_INFO: `/v3/users/0ce42cd5938040f1a4f5925b0238dd68` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:05.646294 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1e8cd-0f76-4870-8f8b-db7b44fb9062 None admin] RBAC: Authorizing `identity:get_user(user_id=0ce42cd5938040f1a4f5925b0238dd68)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:05.648040 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1e8cd-0f76-4870-8f8b-db7b44fb9062 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:05.650844 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 178/366] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:48:05 2022] GET /identity/v3/users/0ce42cd5938040f1a4f5925b0238dd68 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:05.651115 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1e8cd-0f76-4870-8f8b-db7b44fb9062 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:05.657433 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832fe06b-a99e-486a-847b-ad489c5ed696 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:05.713247 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f8779-af1b-4582-b33b-da586b4c5a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:05.714247 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f8779-af1b-4582-b33b-da586b4c5a7d None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:05.714729 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f8779-af1b-4582-b33b-da586b4c5a7d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:05.715213 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f8779-af1b-4582-b33b-da586b4c5a7d None admin] PATH_INFO: `/v3/groups/664a63a166be4fd9a4e70be79f046c5f` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:05.716607 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f8779-af1b-4582-b33b-da586b4c5a7d None admin] RBAC: Authorizing `identity:get_group(group_id=664a63a166be4fd9a4e70be79f046c5f)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:05.718268 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f8779-af1b-4582-b33b-da586b4c5a7d None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:05.720382 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 189/367] 10.222.0.27 () {56 vars in 1180 bytes} [Sun Sep 4 00:48:05 2022] GET /identity/v3/groups/664a63a166be4fd9a4e70be79f046c5f => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:05.721405 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f8779-af1b-4582-b33b-da586b4c5a7d None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:05.721992 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:05.792066 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:05.798126 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1e8cd-0f76-4870-8f8b-db7b44fb9062 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:05.848049 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ce0a3-321f-4f1c-8f3b-2a2923d9296f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:05.849094 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ce0a3-321f-4f1c-8f3b-2a2923d9296f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:05.849584 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ce0a3-321f-4f1c-8f3b-2a2923d9296f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:05.850214 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ce0a3-321f-4f1c-8f3b-2a2923d9296f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:05.851156 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f8779-af1b-4582-b33b-da586b4c5a7d None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:05.851449 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ce0a3-321f-4f1c-8f3b-2a2923d9296f None admin] RBAC: Adding query filter params (group.id=664a63a166be4fd9a4e70be79f046c5f, role.id=d1261c99e1d64f9da63adfff739561e8, scope.project.id=c8fe5139a2a24ec780814c7f39d5b437) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:05.851771 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ce0a3-321f-4f1c-8f3b-2a2923d9296f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:05.853241 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ce0a3-321f-4f1c-8f3b-2a2923d9296f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:05.860186 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 179/368] 10.222.0.27 () {56 vars in 1376 bytes} [Sun Sep 4 00:48:05 2022] GET /identity/v3/role_assignments?group.id=664a63a166be4fd9a4e70be79f046c5f&scope.project.id=c8fe5139a2a24ec780814c7f39d5b437&role.id=d1261c99e1d64f9da63adfff739561e8 => generated 260 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:05.861121 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ce0a3-321f-4f1c-8f3b-2a2923d9296f None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:05.895958 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782cf540-7fc3-42bb-af82-969c82d24b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:05.897074 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782cf540-7fc3-42bb-af82-969c82d24b8d None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:05.897566 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782cf540-7fc3-42bb-af82-969c82d24b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:05.898103 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782cf540-7fc3-42bb-af82-969c82d24b8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:05.979404 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782cf540-7fc3-42bb-af82-969c82d24b8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:05.981360 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782cf540-7fc3-42bb-af82-969c82d24b8d None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:05.986264 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ce0a3-321f-4f1c-8f3b-2a2923d9296f None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:06.002617 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:06.029962 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f524fda-120a-41d5-b6e3-a5eff81f01fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:06.030580 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f524fda-120a-41d5-b6e3-a5eff81f01fd None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:06.031150 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f524fda-120a-41d5-b6e3-a5eff81f01fd None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:06.031281 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f524fda-120a-41d5-b6e3-a5eff81f01fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:06.032220 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f524fda-120a-41d5-b6e3-a5eff81f01fd None admin] RBAC: Adding query filter params (role.id=8f7a2ac695c6482cbf48819cee250c22, user.id=0ce42cd5938040f1a4f5925b0238dd68, scope.system=all) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:06.032437 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f524fda-120a-41d5-b6e3-a5eff81f01fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:06.033930 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f524fda-120a-41d5-b6e3-a5eff81f01fd None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:06.047265 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:06.051931 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 180/369] 10.222.0.27 () {56 vars in 1308 bytes} [Sun Sep 4 00:48:05 2022] GET /identity/v3/role_assignments?user.id=0ce42cd5938040f1a4f5925b0238dd68&scope.system=all&role.id=8f7a2ac695c6482cbf48819cee250c22 => generated 510 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:06.053412 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f524fda-120a-41d5-b6e3-a5eff81f01fd None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:06.128012 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 190/370] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:48:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:48:06.130312 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782cf540-7fc3-42bb-af82-969c82d24b8d None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:06.240847 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f524fda-120a-41d5-b6e3-a5eff81f01fd None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:06.298643 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44368e3-a574-4430-bd33-11647a4f5bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:06.299259 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44368e3-a574-4430-bd33-11647a4f5bab None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:06.299542 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44368e3-a574-4430-bd33-11647a4f5bab None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:06.299813 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44368e3-a574-4430-bd33-11647a4f5bab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:06.300712 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44368e3-a574-4430-bd33-11647a4f5bab None admin] RBAC: Adding query filter params (group.id=664a63a166be4fd9a4e70be79f046c5f, role.id=d1261c99e1d64f9da63adfff739561e8, scope.project.id=5b12c253c6eb48c9a549a9a8436eced1) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:06.301038 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44368e3-a574-4430-bd33-11647a4f5bab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:06.302490 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44368e3-a574-4430-bd33-11647a4f5bab None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:06.311669 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 181/371] 10.222.0.27 () {56 vars in 1376 bytes} [Sun Sep 4 00:48:06 2022] GET /identity/v3/role_assignments?group.id=664a63a166be4fd9a4e70be79f046c5f&scope.project.id=5b12c253c6eb48c9a549a9a8436eced1&role.id=d1261c99e1d64f9da63adfff739561e8 => generated 260 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:06.312339 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44368e3-a574-4430-bd33-11647a4f5bab None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:06.329849 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782cf540-7fc3-42bb-af82-969c82d24b8d None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:06.384310 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f345b5-d981-465c-8d1e-be501b5a1a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:06.385060 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f345b5-d981-465c-8d1e-be501b5a1a0d None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:06.385341 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f345b5-d981-465c-8d1e-be501b5a1a0d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:06.385622 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f345b5-d981-465c-8d1e-be501b5a1a0d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:06.402618 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:06.404723 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f345b5-d981-465c-8d1e-be501b5a1a0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:06.413553 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-41f345b5-d981-465c-8d1e-be501b5a1a0d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:48:06.413553 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:06.413553 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:06.413553 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:06.413553 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:06.413553 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:06.413553 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:06.413553 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:06.413553 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:06.413553 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:06.413553 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:06.413553 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:06.413553 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:06.413553 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:06.413553 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:06.413553 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:06.491591 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:06.491591 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:06.491591 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:06.491591 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:06.491591 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:06.491591 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:06.491591 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:06.491591 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:06.491591 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:06.491591 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:06.491591 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:06.491591 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:06.491591 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:06.491591 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:06.491591 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:06.491591 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:06.492424 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:06.492424 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:06.492424 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:06.492424 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:06.492424 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:06.492424 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:06.492424 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:06.492424 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:06.492424 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:06.492424 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:06.492424 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:48:06.492424 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:06.492424 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 191/372] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:48:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:06.492424 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f345b5-d981-465c-8d1e-be501b5a1a0d None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:06.493311 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:06.550188 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44368e3-a574-4430-bd33-11647a4f5bab None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:06.600443 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1d781-a83a-47cc-affa-b0dc65a20fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:06.601219 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1d781-a83a-47cc-affa-b0dc65a20fb3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:06.601426 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1d781-a83a-47cc-affa-b0dc65a20fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:06.601701 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1d781-a83a-47cc-affa-b0dc65a20fb3 None admin] PATH_INFO: `/v3/projects/5b12c253c6eb48c9a549a9a8436eced1` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:06.602925 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1d781-a83a-47cc-affa-b0dc65a20fb3 None admin] RBAC: Authorizing `identity:get_project(project_id=5b12c253c6eb48c9a549a9a8436eced1)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:06.605088 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1d781-a83a-47cc-affa-b0dc65a20fb3 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:06.607545 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 182/373] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:48:06 2022] GET /identity/v3/projects/5b12c253c6eb48c9a549a9a8436eced1 => generated 302 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:06.615626 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1d781-a83a-47cc-affa-b0dc65a20fb3 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:06.618683 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f345b5-d981-465c-8d1e-be501b5a1a0d None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:06.678860 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f313d-df3c-4708-818d-5330fa2ea522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:06.684688 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f313d-df3c-4708-818d-5330fa2ea522 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:06.685208 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f313d-df3c-4708-818d-5330fa2ea522 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:06.685701 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f313d-df3c-4708-818d-5330fa2ea522 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:06.686591 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f313d-df3c-4708-818d-5330fa2ea522 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:06.687123 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f313d-df3c-4708-818d-5330fa2ea522 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:06.688769 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f313d-df3c-4708-818d-5330fa2ea522 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:06.710981 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 192/374] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:48:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:06.717080 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f313d-df3c-4708-818d-5330fa2ea522 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:06.786734 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1d781-a83a-47cc-affa-b0dc65a20fb3 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:06.790328 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:06.850748 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8708fc-38cd-44aa-92ab-e40458c90b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:06.851380 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8708fc-38cd-44aa-92ab-e40458c90b77 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:06.851659 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8708fc-38cd-44aa-92ab-e40458c90b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:06.851932 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8708fc-38cd-44aa-92ab-e40458c90b77 None admin] PATH_INFO: `/v3/groups/664a63a166be4fd9a4e70be79f046c5f` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:06.853054 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8708fc-38cd-44aa-92ab-e40458c90b77 None admin] RBAC: Authorizing `identity:get_group(group_id=664a63a166be4fd9a4e70be79f046c5f)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:06.854770 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8708fc-38cd-44aa-92ab-e40458c90b77 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:06.858960 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 183/375] 10.222.0.27 () {56 vars in 1180 bytes} [Sun Sep 4 00:48:06 2022] GET /identity/v3/groups/664a63a166be4fd9a4e70be79f046c5f => generated 226 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:06.865320 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8708fc-38cd-44aa-92ab-e40458c90b77 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:06.875269 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:06.915802 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f313d-df3c-4708-818d-5330fa2ea522 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:06.991656 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a521c-2778-4a47-93bd-822129fe0bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:06.992711 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a521c-2778-4a47-93bd-822129fe0bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:06.993002 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a521c-2778-4a47-93bd-822129fe0bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:06.993279 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a521c-2778-4a47-93bd-822129fe0bb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:06.996059 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8708fc-38cd-44aa-92ab-e40458c90b77 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:07.038010 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac72bbb-f9be-4d96-8203-8545470cb7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:07.038945 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac72bbb-f9be-4d96-8203-8545470cb7a2 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:07.039453 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac72bbb-f9be-4d96-8203-8545470cb7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:07.039910 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac72bbb-f9be-4d96-8203-8545470cb7a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:07.040926 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac72bbb-f9be-4d96-8203-8545470cb7a2 None admin] RBAC: Adding query filter params (group.id=664a63a166be4fd9a4e70be79f046c5f, role.id=7df9ebc56a2a4214b2b64a5941812ef0, scope.project.id=5b12c253c6eb48c9a549a9a8436eced1) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:07.041667 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac72bbb-f9be-4d96-8203-8545470cb7a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:07.043348 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac72bbb-f9be-4d96-8203-8545470cb7a2 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:07.050590 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 184/376] 10.222.0.27 () {56 vars in 1376 bytes} [Sun Sep 4 00:48:06 2022] GET /identity/v3/role_assignments?group.id=664a63a166be4fd9a4e70be79f046c5f&scope.project.id=5b12c253c6eb48c9a549a9a8436eced1&role.id=7df9ebc56a2a4214b2b64a5941812ef0 => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:07.081391 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed8d16-29a1-450b-98c6-29e7b015b6ac None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:07.082611 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed8d16-29a1-450b-98c6-29e7b015b6ac None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:07.083067 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed8d16-29a1-450b-98c6-29e7b015b6ac None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:07.085179 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 185/377] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:07.091348 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d6d3f-0687-4804-ac58-085aa2f362cc None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:07.092093 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d6d3f-0687-4804-ac58-085aa2f362cc None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:07.092717 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d6d3f-0687-4804-ac58-085aa2f362cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:07.119498 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-4d9d6d3f-0687-4804-ac58-085aa2f362cc None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:07.121270 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4d9d6d3f-0687-4804-ac58-085aa2f362cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:07.146965 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a521c-2778-4a47-93bd-822129fe0bb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:07.151217 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a521c-2778-4a47-93bd-822129fe0bb3 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:07.193749 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:07.261980 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 186/378] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 00:48:07.294098 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:07.328663 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 193/379] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:48:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 04 00:48:07.335020 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d6d3f-0687-4804-ac58-085aa2f362cc None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:07.444176 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d6d3f-0687-4804-ac58-085aa2f362cc None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:07.483890 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de29938-abbb-49d0-810a-e5ef4c9f48e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:07.484555 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de29938-abbb-49d0-810a-e5ef4c9f48e0 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:07.484837 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de29938-abbb-49d0-810a-e5ef4c9f48e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:07.485109 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de29938-abbb-49d0-810a-e5ef4c9f48e0 None admin] PATH_INFO: `/v3/projects/a1d8c40dd6c04e8e8fe6b0c2b34723cd` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:07.486234 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de29938-abbb-49d0-810a-e5ef4c9f48e0 None admin] RBAC: Authorizing `identity:get_project(project_id=a1d8c40dd6c04e8e8fe6b0c2b34723cd)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:07.487680 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de29938-abbb-49d0-810a-e5ef4c9f48e0 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:07.489349 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 187/380] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:48:07 2022] GET /identity/v3/projects/a1d8c40dd6c04e8e8fe6b0c2b34723cd => generated 344 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:07.494833 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a521c-2778-4a47-93bd-822129fe0bb3 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:07.626040 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a521c-2778-4a47-93bd-822129fe0bb3 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:07.667567 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f2ce7-1ed6-4fd7-bfb5-ea700f808f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:07.668203 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f2ce7-1ed6-4fd7-bfb5-ea700f808f74 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:07.668525 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f2ce7-1ed6-4fd7-bfb5-ea700f808f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:07.668807 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f2ce7-1ed6-4fd7-bfb5-ea700f808f74 None admin] PATH_INFO: `/v3/groups/fcbc87e1a8a24d9abda6502eefa59888` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:07.673930 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f2ce7-1ed6-4fd7-bfb5-ea700f808f74 None admin] RBAC: Authorizing `identity:get_group(group_id=fcbc87e1a8a24d9abda6502eefa59888)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:07.675498 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f2ce7-1ed6-4fd7-bfb5-ea700f808f74 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:07.677357 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 194/381] 10.222.0.27 () {56 vars in 1180 bytes} [Sun Sep 4 00:48:07 2022] GET /identity/v3/groups/fcbc87e1a8a24d9abda6502eefa59888 => generated 229 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:07.683038 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de29938-abbb-49d0-810a-e5ef4c9f48e0 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:07.722330 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:07.793824 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de29938-abbb-49d0-810a-e5ef4c9f48e0 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:07.808599 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f2ce7-1ed6-4fd7-bfb5-ea700f808f74 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:07.837283 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3572ab6-30b6-42f7-8a73-632519ce0581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:07.837892 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3572ab6-30b6-42f7-8a73-632519ce0581 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:07.838177 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3572ab6-30b6-42f7-8a73-632519ce0581 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:07.838977 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3572ab6-30b6-42f7-8a73-632519ce0581 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:07.839728 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3572ab6-30b6-42f7-8a73-632519ce0581 None admin] RBAC: Adding query filter params (group.id=fcbc87e1a8a24d9abda6502eefa59888, role.id=c8a59d34ee2f4cefa6940cac778260c7, scope.project.id=a1d8c40dd6c04e8e8fe6b0c2b34723cd) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:07.840037 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3572ab6-30b6-42f7-8a73-632519ce0581 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:07.841842 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3572ab6-30b6-42f7-8a73-632519ce0581 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:07.850638 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 188/382] 10.222.0.27 () {56 vars in 1376 bytes} [Sun Sep 4 00:48:07 2022] GET /identity/v3/role_assignments?group.id=fcbc87e1a8a24d9abda6502eefa59888&scope.project.id=a1d8c40dd6c04e8e8fe6b0c2b34723cd&role.id=c8a59d34ee2f4cefa6940cac778260c7 => generated 260 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:07.869575 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:07.940182 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f2ce7-1ed6-4fd7-bfb5-ea700f808f74 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:07.980542 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f402a2-cabb-4a87-8c90-6f50cf43cb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:07.981449 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f402a2-cabb-4a87-8c90-6f50cf43cb57 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:07.981889 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f402a2-cabb-4a87-8c90-6f50cf43cb57 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:07.982306 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f402a2-cabb-4a87-8c90-6f50cf43cb57 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:07.995001 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f402a2-cabb-4a87-8c90-6f50cf43cb57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:08.000371 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-78f402a2-cabb-4a87-8c90-6f50cf43cb57 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 00:48:08.000371 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:08.000371 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:08.000371 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:08.000371 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:08.000371 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:08.000371 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:08.000371 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:08.000371 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:08.000371 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:08.000371 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:08.000371 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:08.000371 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:08.000371 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:08.000371 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:08.000371 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:08.001513 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:08.001513 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:08.001513 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:08.001513 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:08.001513 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:08.001513 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:08.001513 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:08.001513 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:08.001513 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:08.001513 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:08.001513 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:08.001513 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:08.001513 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:08.001513 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:08.001513 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:08.001513 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:08.002498 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:08.002498 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:08.002498 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:08.002498 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:08.002498 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:08.002498 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:08.002498 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:08.002498 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:08.002498 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:08.002498 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:08.002498 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 00:48:08.002498 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:08.008601 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 195/383] 10.222.0.27 () {56 vars in 1114 bytes} [Sun Sep 4 00:48:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:08.010989 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3572ab6-30b6-42f7-8a73-632519ce0581 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:08.131358 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:08.136083 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3572ab6-30b6-42f7-8a73-632519ce0581 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:08.182642 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e8bf5c-734a-478b-be98-b7ca3bc2de1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:08.183491 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8bf5c-734a-478b-be98-b7ca3bc2de1d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:08.183941 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8bf5c-734a-478b-be98-b7ca3bc2de1d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:08.184380 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8bf5c-734a-478b-be98-b7ca3bc2de1d None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:08.185283 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e8bf5c-734a-478b-be98-b7ca3bc2de1d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:08.185731 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e8bf5c-734a-478b-be98-b7ca3bc2de1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:08.187326 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e8bf5c-734a-478b-be98-b7ca3bc2de1d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:08.198053 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 189/384] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:48:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:08.203766 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f402a2-cabb-4a87-8c90-6f50cf43cb57 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:08.325424 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f402a2-cabb-4a87-8c90-6f50cf43cb57 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:08.327163 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:08.364683 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63cf6ba-a9c9-43be-b8ef-ad8411a634bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:08.366916 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63cf6ba-a9c9-43be-b8ef-ad8411a634bc None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:08.367203 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63cf6ba-a9c9-43be-b8ef-ad8411a634bc None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:08.367482 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63cf6ba-a9c9-43be-b8ef-ad8411a634bc None admin] PATH_INFO: `/v3/groups/664a63a166be4fd9a4e70be79f046c5f` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:08.368685 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63cf6ba-a9c9-43be-b8ef-ad8411a634bc None admin] RBAC: Authorizing `identity:get_group(group_id=664a63a166be4fd9a4e70be79f046c5f)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:08.370202 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63cf6ba-a9c9-43be-b8ef-ad8411a634bc None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:08.372151 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 196/385] 10.222.0.27 () {56 vars in 1180 bytes} [Sun Sep 4 00:48:08 2022] GET /identity/v3/groups/664a63a166be4fd9a4e70be79f046c5f => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:08.377297 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e8bf5c-734a-478b-be98-b7ca3bc2de1d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:08.507109 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4229e17-0f01-41d2-b36b-2a5d1d271a91 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:08.507453 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4229e17-0f01-41d2-b36b-2a5d1d271a91 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:08.507669 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4229e17-0f01-41d2-b36b-2a5d1d271a91 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:08.509368 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 197/386] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:08.511096 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e8bf5c-734a-478b-be98-b7ca3bc2de1d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:08.514568 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff48d1-f8fe-404e-abdb-95f0253ae9bf None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:08.515085 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff48d1-f8fe-404e-abdb-95f0253ae9bf None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:08.515577 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff48d1-f8fe-404e-abdb-95f0253ae9bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:08.535948 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-ccff48d1-f8fe-404e-abdb-95f0253ae9bf None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:08.537564 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ccff48d1-f8fe-404e-abdb-95f0253ae9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:08.562665 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49f0f2-aaa5-43dc-82da-b02e996afc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:08.563309 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49f0f2-aaa5-43dc-82da-b02e996afc8f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:08.563592 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49f0f2-aaa5-43dc-82da-b02e996afc8f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:08.563867 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49f0f2-aaa5-43dc-82da-b02e996afc8f None admin] PATH_INFO: `/v3/projects/c8fe5139a2a24ec780814c7f39d5b437` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:08.567742 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee49f0f2-aaa5-43dc-82da-b02e996afc8f None admin] RBAC: Authorizing `identity:get_project(project_id=c8fe5139a2a24ec780814c7f39d5b437)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:08.569277 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee49f0f2-aaa5-43dc-82da-b02e996afc8f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:08.575474 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 190/387] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:48:08 2022] GET /identity/v3/projects/c8fe5139a2a24ec780814c7f39d5b437 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:08.580887 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49f0f2-aaa5-43dc-82da-b02e996afc8f None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:08.638654 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:08.663897 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 198/388] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:08.728432 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49f0f2-aaa5-43dc-82da-b02e996afc8f None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:08.769329 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be29eac-7fcb-40a0-8624-f366e831b634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:08.770156 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be29eac-7fcb-40a0-8624-f366e831b634 None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:08.770446 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be29eac-7fcb-40a0-8624-f366e831b634 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:08.770774 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be29eac-7fcb-40a0-8624-f366e831b634 None admin] PATH_INFO: `/v3/projects/c8fe5139a2a24ec780814c7f39d5b437/groups/664a63a166be4fd9a4e70be79f046c5f/roles/7df9ebc56a2a4214b2b64a5941812ef0` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:08.773073 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be29eac-7fcb-40a0-8624-f366e831b634 None admin] RBAC: Authorizing `identity:create_grant(project_id=c8fe5139a2a24ec780814c7f39d5b437, group_id=664a63a166be4fd9a4e70be79f046c5f, role_id=7df9ebc56a2a4214b2b64a5941812ef0)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:08.774500 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be29eac-7fcb-40a0-8624-f366e831b634 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:08.852077 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 191/389] 10.222.0.27 () {58 vars in 1442 bytes} [Sun Sep 4 00:48:08 2022] PUT /identity/v3/projects/c8fe5139a2a24ec780814c7f39d5b437/groups/664a63a166be4fd9a4e70be79f046c5f/roles/7df9ebc56a2a4214b2b64a5941812ef0 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 00:48:09.029557 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff48d1-f8fe-404e-abdb-95f0253ae9bf None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:09.141070 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:09.144962 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff48d1-f8fe-404e-abdb-95f0253ae9bf None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:09.183557 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c2cab-18b4-4a81-a183-576267972f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:09.184265 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c2cab-18b4-4a81-a183-576267972f0d None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:09.184576 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c2cab-18b4-4a81-a183-576267972f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:09.184861 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c2cab-18b4-4a81-a183-576267972f0d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:09.196424 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c2cab-18b4-4a81-a183-576267972f0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:09.202293 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-670c2cab-18b4-4a81-a183-576267972f0d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:48:09.202293 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:09.202293 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:09.202293 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:09.202293 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:09.202293 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:09.202293 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:09.202293 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:09.202293 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:09.202293 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:09.202293 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:09.202293 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:09.202293 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:09.202293 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:09.202293 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:09.202293 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:09.203173 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:09.203173 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:09.203173 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:09.203173 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:09.203173 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:09.203173 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:09.203173 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:09.203173 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:09.203173 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:09.203173 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:09.203173 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:09.203173 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:09.203173 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:09.203173 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:09.203173 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:09.203173 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:09.203958 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:09.203958 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:09.203958 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:09.203958 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:09.203958 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:09.203958 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:09.203958 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:09.203958 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:09.203958 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:09.203958 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:09.203958 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:48:09.203958 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:09.208611 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 199/390] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:48:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:09.210662 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be29eac-7fcb-40a0-8624-f366e831b634 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:09.308894 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:09.322487 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be29eac-7fcb-40a0-8624-f366e831b634 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:09.351798 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759d2d9-440d-4329-a1c5-fb16f02d3b59 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:09.352416 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759d2d9-440d-4329-a1c5-fb16f02d3b59 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:09.352974 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759d2d9-440d-4329-a1c5-fb16f02d3b59 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:09.354756 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 200/391] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 00:48:09.359908 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e55ba-b4ad-4f51-92cf-7dd7e49b4944 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:09.360469 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e55ba-b4ad-4f51-92cf-7dd7e49b4944 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:09.360958 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e55ba-b4ad-4f51-92cf-7dd7e49b4944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:09.361611 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9208e2-d6eb-4242-96de-e77b148e6c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:09.362236 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9208e2-d6eb-4242-96de-e77b148e6c77 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:09.362521 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9208e2-d6eb-4242-96de-e77b148e6c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:09.362833 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9208e2-d6eb-4242-96de-e77b148e6c77 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:09.363513 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9208e2-d6eb-4242-96de-e77b148e6c77 None admin] RBAC: Adding query filter params (name=member) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:09.363818 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9208e2-d6eb-4242-96de-e77b148e6c77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:09.365211 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9208e2-d6eb-4242-96de-e77b148e6c77 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:09.378951 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 192/392] 10.222.0.27 () {56 vars in 1101 bytes} [Sun Sep 4 00:48:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:09.390776 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9208e2-d6eb-4242-96de-e77b148e6c77 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:09.398161 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-196e55ba-b4ad-4f51-92cf-7dd7e49b4944 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:09.399652 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-196e55ba-b4ad-4f51-92cf-7dd7e49b4944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:09.522371 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 201/393] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:09.523662 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9208e2-d6eb-4242-96de-e77b148e6c77 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:09.535958 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b06b14-eee7-48ec-9116-687cbd7af940 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:09.536261 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b06b14-eee7-48ec-9116-687cbd7af940 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:09.536550 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b06b14-eee7-48ec-9116-687cbd7af940 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:09.537912 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 202/394] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 00:48:09.552413 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6400f2a-7082-49e6-bc0d-03d6ef900dda None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:09.552740 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6400f2a-7082-49e6-bc0d-03d6ef900dda None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:09.553006 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6400f2a-7082-49e6-bc0d-03d6ef900dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:09.573406 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb608d50-c8ef-43a4-a19e-c14a14b286b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:09.574383 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb608d50-c8ef-43a4-a19e-c14a14b286b0 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:09.574862 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb608d50-c8ef-43a4-a19e-c14a14b286b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:09.575332 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb608d50-c8ef-43a4-a19e-c14a14b286b0 None admin] PATH_INFO: `/v3/groups/664a63a166be4fd9a4e70be79f046c5f` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:09.578902 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb608d50-c8ef-43a4-a19e-c14a14b286b0 None admin] RBAC: Authorizing `identity:get_group(group_id=664a63a166be4fd9a4e70be79f046c5f)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:09.579877 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-c6400f2a-7082-49e6-bc0d-03d6ef900dda None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:09.581110 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb608d50-c8ef-43a4-a19e-c14a14b286b0 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:09.581234 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c6400f2a-7082-49e6-bc0d-03d6ef900dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:09.583519 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 193/395] 10.222.0.27 () {56 vars in 1180 bytes} [Sun Sep 4 00:48:09 2022] GET /identity/v3/groups/664a63a166be4fd9a4e70be79f046c5f => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:48:09.601411 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb608d50-c8ef-43a4-a19e-c14a14b286b0 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:09.629866 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:09.685766 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 203/396] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:09.724837 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb608d50-c8ef-43a4-a19e-c14a14b286b0 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:09.772249 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9e4eb-f826-49b7-88ce-45833a5e038c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:09.773701 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9e4eb-f826-49b7-88ce-45833a5e038c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:09.774203 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9e4eb-f826-49b7-88ce-45833a5e038c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:09.774688 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9e4eb-f826-49b7-88ce-45833a5e038c None admin] PATH_INFO: `/v3/projects/c8fe5139a2a24ec780814c7f39d5b437` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:09.776100 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e9e4eb-f826-49b7-88ce-45833a5e038c None admin] RBAC: Authorizing `identity:get_project(project_id=c8fe5139a2a24ec780814c7f39d5b437)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:09.777778 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e9e4eb-f826-49b7-88ce-45833a5e038c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:09.779946 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 194/397] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:48:09 2022] GET /identity/v3/projects/c8fe5139a2a24ec780814c7f39d5b437 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:48:09.786276 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6400f2a-7082-49e6-bc0d-03d6ef900dda None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:09.820190 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:09.884939 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9e4eb-f826-49b7-88ce-45833a5e038c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:09.920266 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6400f2a-7082-49e6-bc0d-03d6ef900dda None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:09.959644 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c438e3-012b-4afe-b626-551479a694cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:09.960325 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c438e3-012b-4afe-b626-551479a694cc None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:09.960634 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c438e3-012b-4afe-b626-551479a694cc None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:09.960911 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c438e3-012b-4afe-b626-551479a694cc None admin] PATH_INFO: `/v3/projects/c8fe5139a2a24ec780814c7f39d5b437/groups/664a63a166be4fd9a4e70be79f046c5f/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:09.964108 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c438e3-012b-4afe-b626-551479a694cc None admin] RBAC: Authorizing `identity:create_grant(project_id=c8fe5139a2a24ec780814c7f39d5b437, group_id=664a63a166be4fd9a4e70be79f046c5f, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:09.965585 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c438e3-012b-4afe-b626-551479a694cc None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:10.028288 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:10.043674 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9e4eb-f826-49b7-88ce-45833a5e038c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:10.070466 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 204/398] 10.222.0.27 () {58 vars in 1442 bytes} [Sun Sep 4 00:48:09 2022] PUT /identity/v3/projects/c8fe5139a2a24ec780814c7f39d5b437/groups/664a63a166be4fd9a4e70be79f046c5f/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 00:48:10.072550 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c438e3-012b-4afe-b626-551479a694cc None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:10.082278 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6da5d-a1c6-4386-bf84-c775e3ee6754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:10.082926 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6da5d-a1c6-4386-bf84-c775e3ee6754 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:10.083206 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6da5d-a1c6-4386-bf84-c775e3ee6754 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:10.083478 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6da5d-a1c6-4386-bf84-c775e3ee6754 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:10.113536 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6da5d-a1c6-4386-bf84-c775e3ee6754 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:10.118604 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-5cd6da5d-a1c6-4386-bf84-c775e3ee6754 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:48:10.118604 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:10.118604 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:10.118604 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:10.118604 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:10.118604 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:10.118604 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:10.118604 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:10.118604 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:10.118604 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:10.118604 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:10.118604 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:10.118604 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:10.118604 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:10.118604 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:10.118604 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:10.119572 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:10.119572 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:10.119572 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:10.119572 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:10.119572 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:10.119572 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:10.119572 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:10.119572 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:10.119572 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:10.119572 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:10.119572 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:10.119572 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:10.119572 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:10.119572 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:10.119572 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:10.119572 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:10.120351 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:10.120351 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:10.120351 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:10.120351 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:10.120351 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:10.120351 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:10.120351 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:10.120351 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:10.120351 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:10.120351 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:10.120351 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:48:10.120351 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:10.120351 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 195/399] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:48:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:10.129687 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6da5d-a1c6-4386-bf84-c775e3ee6754 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:10.212873 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:10.233141 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c438e3-012b-4afe-b626-551479a694cc None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:10.275073 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94feade7-a644-4f39-9b45-ec5474f251ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:10.276072 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94feade7-a644-4f39-9b45-ec5474f251ce None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:10.276391 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94feade7-a644-4f39-9b45-ec5474f251ce None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:10.276729 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94feade7-a644-4f39-9b45-ec5474f251ce None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:10.278500 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6da5d-a1c6-4386-bf84-c775e3ee6754 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:10.291569 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94feade7-a644-4f39-9b45-ec5474f251ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:10.298323 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-94feade7-a644-4f39-9b45-ec5474f251ce None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 00:48:10.298323 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:10.298323 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:10.298323 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:10.298323 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:10.298323 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:10.298323 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:10.298323 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:10.298323 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:10.298323 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:10.298323 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:10.298323 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:10.298323 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:10.298323 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:10.298323 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:10.298323 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:10.299525 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:10.299525 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:10.299525 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:10.299525 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:10.299525 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:10.299525 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:10.299525 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:10.299525 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:10.299525 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:10.299525 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:10.299525 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:10.299525 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:10.299525 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:10.299525 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:10.299525 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:10.299525 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:10.300552 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:10.300552 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:10.300552 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:10.300552 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:10.300552 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:10.300552 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:10.300552 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:10.300552 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:10.300552 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:10.300552 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:10.300552 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 00:48:10.300552 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:10.300552 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 205/400] 10.222.0.27 () {56 vars in 1114 bytes} [Sun Sep 4 00:48:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:10.301861 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5899ed-40ce-4fb8-a01b-c5bd45376bf4 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:10.301967 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5899ed-40ce-4fb8-a01b-c5bd45376bf4 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:10.303560 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5899ed-40ce-4fb8-a01b-c5bd45376bf4 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:10.303560 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 206/401] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:10.307789 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5899ed-40ce-4fb8-a01b-c5bd45376bf4 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:10.323293 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26a008-418e-4f20-91d4-df8ffa3d0d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:10.324212 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26a008-418e-4f20-91d4-df8ffa3d0d26 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:10.324725 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26a008-418e-4f20-91d4-df8ffa3d0d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:10.325310 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26a008-418e-4f20-91d4-df8ffa3d0d26 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:10.326041 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d26a008-418e-4f20-91d4-df8ffa3d0d26 None admin] RBAC: Adding query filter params (name=member) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:10.326481 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d26a008-418e-4f20-91d4-df8ffa3d0d26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:10.327997 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d26a008-418e-4f20-91d4-df8ffa3d0d26 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:10.339545 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 196/402] 10.222.0.27 () {56 vars in 1101 bytes} [Sun Sep 4 00:48:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:10.344127 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80a26c-613b-4651-b66d-d424041c019e None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:10.344127 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80a26c-613b-4651-b66d-d424041c019e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:10.344127 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80a26c-613b-4651-b66d-d424041c019e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:10.366857 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-0a80a26c-613b-4651-b66d-d424041c019e None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:10.368054 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0a80a26c-613b-4651-b66d-d424041c019e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:10.396815 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:10.443730 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 197/403] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:10.445835 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a80a26c-613b-4651-b66d-d424041c019e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:10.454000 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5899ed-40ce-4fb8-a01b-c5bd45376bf4 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:10.493533 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b2b38-a4a0-411a-9d26-e025a5198ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:10.494280 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b2b38-a4a0-411a-9d26-e025a5198ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:10.494623 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b2b38-a4a0-411a-9d26-e025a5198ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:10.494997 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b2b38-a4a0-411a-9d26-e025a5198ec7 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:10.495726 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b2b38-a4a0-411a-9d26-e025a5198ec7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:10.496090 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b2b38-a4a0-411a-9d26-e025a5198ec7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:10.497614 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b2b38-a4a0-411a-9d26-e025a5198ec7 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:10.507362 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 207/404] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:48:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:10.513011 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b2b38-a4a0-411a-9d26-e025a5198ec7 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:10.516010 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:10.583176 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a80a26c-613b-4651-b66d-d424041c019e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:10.624127 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36db3b-7908-4fb1-a938-dbf57347909c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:10.625628 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36db3b-7908-4fb1-a938-dbf57347909c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:10.626113 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36db3b-7908-4fb1-a938-dbf57347909c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:10.626575 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36db3b-7908-4fb1-a938-dbf57347909c None admin] PATH_INFO: `/v3/groups/664a63a166be4fd9a4e70be79f046c5f` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:10.628174 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b36db3b-7908-4fb1-a938-dbf57347909c None admin] RBAC: Authorizing `identity:get_group(group_id=664a63a166be4fd9a4e70be79f046c5f)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:10.629969 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b36db3b-7908-4fb1-a938-dbf57347909c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:10.632710 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 198/405] 10.222.0.27 () {56 vars in 1180 bytes} [Sun Sep 4 00:48:10 2022] GET /identity/v3/groups/664a63a166be4fd9a4e70be79f046c5f => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:10.641311 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36db3b-7908-4fb1-a938-dbf57347909c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:10.653484 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:10.686108 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b2b38-a4a0-411a-9d26-e025a5198ec7 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:10.748072 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd1583-cc64-4639-ad5c-7d610523cdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:10.749071 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd1583-cc64-4639-ad5c-7d610523cdf2 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:10.750404 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd1583-cc64-4639-ad5c-7d610523cdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:10.752028 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd1583-cc64-4639-ad5c-7d610523cdf2 None admin] PATH_INFO: `/v3/groups/664a63a166be4fd9a4e70be79f046c5f` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:10.755075 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd1583-cc64-4639-ad5c-7d610523cdf2 None admin] RBAC: Authorizing `identity:get_group(group_id=664a63a166be4fd9a4e70be79f046c5f)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:10.756749 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd1583-cc64-4639-ad5c-7d610523cdf2 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:10.759836 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 208/406] 10.222.0.27 () {56 vars in 1180 bytes} [Sun Sep 4 00:48:10 2022] GET /identity/v3/groups/664a63a166be4fd9a4e70be79f046c5f => generated 226 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:10.765666 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd1583-cc64-4639-ad5c-7d610523cdf2 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:10.809391 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36db3b-7908-4fb1-a938-dbf57347909c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:10.850855 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6277650-36a3-44e9-8dc0-5102267d8257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:10.851505 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6277650-36a3-44e9-8dc0-5102267d8257 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:10.851781 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6277650-36a3-44e9-8dc0-5102267d8257 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:10.852055 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6277650-36a3-44e9-8dc0-5102267d8257 None admin] PATH_INFO: `/v3/projects/5b12c253c6eb48c9a549a9a8436eced1` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:10.853182 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6277650-36a3-44e9-8dc0-5102267d8257 None admin] RBAC: Authorizing `identity:get_project(project_id=5b12c253c6eb48c9a549a9a8436eced1)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:10.854711 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6277650-36a3-44e9-8dc0-5102267d8257 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:10.861238 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 199/407] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:48:10 2022] GET /identity/v3/projects/5b12c253c6eb48c9a549a9a8436eced1 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:48:10.862066 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6277650-36a3-44e9-8dc0-5102267d8257 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:10.865757 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:10.893326 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd1583-cc64-4639-ad5c-7d610523cdf2 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:10.935504 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20444fd1-840a-423e-92aa-b2bad609967f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:10.936568 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20444fd1-840a-423e-92aa-b2bad609967f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:10.936855 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20444fd1-840a-423e-92aa-b2bad609967f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:10.937127 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20444fd1-840a-423e-92aa-b2bad609967f None admin] PATH_INFO: `/v3/projects/5b12c253c6eb48c9a549a9a8436eced1` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:10.938834 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20444fd1-840a-423e-92aa-b2bad609967f None admin] RBAC: Authorizing `identity:get_project(project_id=5b12c253c6eb48c9a549a9a8436eced1)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:10.940647 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20444fd1-840a-423e-92aa-b2bad609967f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:10.943563 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 209/408] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:48:10 2022] GET /identity/v3/projects/5b12c253c6eb48c9a549a9a8436eced1 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:10.944323 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20444fd1-840a-423e-92aa-b2bad609967f None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:10.949186 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:10.976618 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6277650-36a3-44e9-8dc0-5102267d8257 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:11.012517 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70260e7-1c23-4653-8d8f-1f3478950645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:11.013551 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70260e7-1c23-4653-8d8f-1f3478950645 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:11.013907 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70260e7-1c23-4653-8d8f-1f3478950645 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:11.014252 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70260e7-1c23-4653-8d8f-1f3478950645 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:11.025760 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70260e7-1c23-4653-8d8f-1f3478950645 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:11.031396 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-f70260e7-1c23-4653-8d8f-1f3478950645 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:48:11.031396 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:11.031396 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:11.031396 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:11.031396 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:11.031396 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:11.031396 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:11.031396 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:11.031396 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:11.031396 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:11.031396 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:11.031396 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:11.031396 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:11.031396 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:11.031396 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:11.031396 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:11.032250 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:11.032250 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:11.032250 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:11.032250 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:11.032250 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:11.032250 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:11.032250 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:11.032250 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:11.032250 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:11.032250 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:11.032250 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:11.032250 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:11.032250 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:11.032250 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:11.032250 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:11.032250 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:11.033099 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:11.033099 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:11.033099 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:11.033099 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:11.033099 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:11.033099 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:11.033099 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:11.033099 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:11.033099 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:11.033099 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:11.033099 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:48:11.033099 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:11.037925 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 200/409] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:48:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:11.038620 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70260e7-1c23-4653-8d8f-1f3478950645 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:11.079138 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20444fd1-840a-423e-92aa-b2bad609967f None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:11.124802 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a3241-ec11-4ba8-ae93-f53a77afa9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:11.126386 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a3241-ec11-4ba8-ae93-f53a77afa9ac None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:11.126865 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a3241-ec11-4ba8-ae93-f53a77afa9ac None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:11.127566 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a3241-ec11-4ba8-ae93-f53a77afa9ac None admin] PATH_INFO: `/v3/projects/5b12c253c6eb48c9a549a9a8436eced1/groups/664a63a166be4fd9a4e70be79f046c5f/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:11.130463 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3a3241-ec11-4ba8-ae93-f53a77afa9ac None admin] RBAC: Authorizing `identity:create_grant(project_id=5b12c253c6eb48c9a549a9a8436eced1, group_id=664a63a166be4fd9a4e70be79f046c5f, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:11.132774 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3a3241-ec11-4ba8-ae93-f53a77afa9ac None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:11.145739 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70260e7-1c23-4653-8d8f-1f3478950645 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:11.182268 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f1a1fa-8cd7-43d5-8885-3ccca9f70b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:11.182925 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1a1fa-8cd7-43d5-8885-3ccca9f70b28 None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:11.183206 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1a1fa-8cd7-43d5-8885-3ccca9f70b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:11.183481 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1a1fa-8cd7-43d5-8885-3ccca9f70b28 None admin] PATH_INFO: `/v3/projects/5b12c253c6eb48c9a549a9a8436eced1/groups/664a63a166be4fd9a4e70be79f046c5f/roles/7df9ebc56a2a4214b2b64a5941812ef0` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:11.185613 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f1a1fa-8cd7-43d5-8885-3ccca9f70b28 None admin] RBAC: Authorizing `identity:create_grant(project_id=5b12c253c6eb48c9a549a9a8436eced1, group_id=664a63a166be4fd9a4e70be79f046c5f, role_id=7df9ebc56a2a4214b2b64a5941812ef0)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:11.187148 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f1a1fa-8cd7-43d5-8885-3ccca9f70b28 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:11.189638 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:11.235813 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:11.248500 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 210/410] 10.222.0.27 () {58 vars in 1442 bytes} [Sun Sep 4 00:48:10 2022] PUT /identity/v3/projects/5b12c253c6eb48c9a549a9a8436eced1/groups/664a63a166be4fd9a4e70be79f046c5f/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 00:48:11.248500 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a3241-ec11-4ba8-ae93-f53a77afa9ac None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:11.285642 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 201/411] 10.222.0.27 () {58 vars in 1442 bytes} [Sun Sep 4 00:48:11 2022] PUT /identity/v3/projects/5b12c253c6eb48c9a549a9a8436eced1/groups/664a63a166be4fd9a4e70be79f046c5f/roles/7df9ebc56a2a4214b2b64a5941812ef0 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 00:48:11.406786 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a3241-ec11-4ba8-ae93-f53a77afa9ac None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:11.413417 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fca21c-f562-447f-97b7-e113d0310c71 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:11.413772 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fca21c-f562-447f-97b7-e113d0310c71 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:11.413992 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fca21c-f562-447f-97b7-e113d0310c71 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:11.415744 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 202/412] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:11.427731 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cde253-ae54-4aa9-9599-6f02b4b8d1f2 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:11.428034 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cde253-ae54-4aa9-9599-6f02b4b8d1f2 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:11.428278 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cde253-ae54-4aa9-9599-6f02b4b8d1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:11.457906 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7edc4b-ef19-4de6-af68-702555716ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:11.459045 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7edc4b-ef19-4de6-af68-702555716ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:11.459525 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7edc4b-ef19-4de6-af68-702555716ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:11.459628 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-93cde253-ae54-4aa9-9599-6f02b4b8d1f2 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:11.460109 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7edc4b-ef19-4de6-af68-702555716ee4 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:11.460839 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-93cde253-ae54-4aa9-9599-6f02b4b8d1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:11.461231 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7edc4b-ef19-4de6-af68-702555716ee4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:11.461797 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7edc4b-ef19-4de6-af68-702555716ee4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:11.463700 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7edc4b-ef19-4de6-af68-702555716ee4 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:11.476548 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 211/413] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:48:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:11.482166 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7edc4b-ef19-4de6-af68-702555716ee4 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:11.557084 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 203/414] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:11.572156 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:11.610465 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7edc4b-ef19-4de6-af68-702555716ee4 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:11.646972 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66556256-18b1-49ab-9d1b-5379b347cd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:11.647603 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66556256-18b1-49ab-9d1b-5379b347cd4b None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:11.647880 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66556256-18b1-49ab-9d1b-5379b347cd4b None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:11.648150 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66556256-18b1-49ab-9d1b-5379b347cd4b None admin] PATH_INFO: `/v3/groups/fcbc87e1a8a24d9abda6502eefa59888` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:11.649297 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66556256-18b1-49ab-9d1b-5379b347cd4b None admin] RBAC: Authorizing `identity:get_group(group_id=fcbc87e1a8a24d9abda6502eefa59888)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:11.650743 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66556256-18b1-49ab-9d1b-5379b347cd4b None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:11.652617 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 212/415] 10.222.0.27 () {56 vars in 1180 bytes} [Sun Sep 4 00:48:11 2022] GET /identity/v3/groups/fcbc87e1a8a24d9abda6502eefa59888 => generated 229 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:11.661545 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cde253-ae54-4aa9-9599-6f02b4b8d1f2 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:11.775365 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cde253-ae54-4aa9-9599-6f02b4b8d1f2 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:11.803161 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:11.830907 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba5e6e1-696e-4743-b623-aafd80cdca00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:11.830907 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5e6e1-696e-4743-b623-aafd80cdca00 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:11.830907 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5e6e1-696e-4743-b623-aafd80cdca00 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:11.830907 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5e6e1-696e-4743-b623-aafd80cdca00 None admin] PATH_INFO: `/v3/projects/a1d8c40dd6c04e8e8fe6b0c2b34723cd` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:11.832563 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba5e6e1-696e-4743-b623-aafd80cdca00 None admin] RBAC: Authorizing `identity:get_project(project_id=a1d8c40dd6c04e8e8fe6b0c2b34723cd)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:11.832563 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba5e6e1-696e-4743-b623-aafd80cdca00 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:11.832563 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 204/416] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:48:11 2022] GET /identity/v3/projects/a1d8c40dd6c04e8e8fe6b0c2b34723cd => generated 344 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:11.832761 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66556256-18b1-49ab-9d1b-5379b347cd4b None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:11.905725 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba5e6e1-696e-4743-b623-aafd80cdca00 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:12.002153 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66556256-18b1-49ab-9d1b-5379b347cd4b None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:12.031328 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:12.039990 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd984d-ef09-449b-a186-9ed0164b1986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:12.040686 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd984d-ef09-449b-a186-9ed0164b1986 None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:12.041221 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd984d-ef09-449b-a186-9ed0164b1986 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:12.041499 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd984d-ef09-449b-a186-9ed0164b1986 None admin] PATH_INFO: `/v3/projects/a1d8c40dd6c04e8e8fe6b0c2b34723cd/groups/fcbc87e1a8a24d9abda6502eefa59888/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:12.043407 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffd984d-ef09-449b-a186-9ed0164b1986 None admin] RBAC: Authorizing `identity:create_grant(project_id=a1d8c40dd6c04e8e8fe6b0c2b34723cd, group_id=fcbc87e1a8a24d9abda6502eefa59888, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:12.045107 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffd984d-ef09-449b-a186-9ed0164b1986 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:12.059262 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba5e6e1-696e-4743-b623-aafd80cdca00 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:12.101305 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25cb57-b977-469d-9153-034444a0b31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:12.101886 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25cb57-b977-469d-9153-034444a0b31e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:12.102305 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25cb57-b977-469d-9153-034444a0b31e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:12.102630 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25cb57-b977-469d-9153-034444a0b31e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:12.114441 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d25cb57-b977-469d-9153-034444a0b31e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:12.120057 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-2d25cb57-b977-469d-9153-034444a0b31e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 00:48:12.120057 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:12.120057 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:12.120057 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:12.120057 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:12.120057 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:12.120057 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:12.120057 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:12.120057 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:12.120057 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:12.120057 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:12.120057 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:12.120057 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:12.120057 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:12.120057 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:12.120057 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:12.121059 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:12.121059 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:12.121059 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:12.121059 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:12.121059 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:12.121059 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:12.121059 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:12.121059 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:12.121059 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:12.121059 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:12.121059 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:12.121059 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:12.121059 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:12.121059 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:12.121059 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:12.121059 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:12.121839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:12.121839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:12.121839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:12.121839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:12.121839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:12.121839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:12.121839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:12.121839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:12.121839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:12.121839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:12.121839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 00:48:12.121839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:12.121839 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 205/417] 10.222.0.27 () {56 vars in 1114 bytes} [Sun Sep 4 00:48:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:12.126806 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25cb57-b977-469d-9153-034444a0b31e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:12.146585 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 213/418] 10.222.0.27 () {58 vars in 1442 bytes} [Sun Sep 4 00:48:11 2022] PUT /identity/v3/projects/a1d8c40dd6c04e8e8fe6b0c2b34723cd/groups/fcbc87e1a8a24d9abda6502eefa59888/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 00:48:12.189440 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:12.243956 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25cb57-b977-469d-9153-034444a0b31e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:12.283223 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd95c8d8-b2d3-408e-8b0f-ebc1c10002d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:12.283846 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95c8d8-b2d3-408e-8b0f-ebc1c10002d6 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:12.284125 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95c8d8-b2d3-408e-8b0f-ebc1c10002d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:12.284979 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95c8d8-b2d3-408e-8b0f-ebc1c10002d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:12.285667 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd95c8d8-b2d3-408e-8b0f-ebc1c10002d6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:12.285975 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd95c8d8-b2d3-408e-8b0f-ebc1c10002d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:12.287432 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd95c8d8-b2d3-408e-8b0f-ebc1c10002d6 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:12.297880 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 206/419] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:48:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:48:12.303644 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd984d-ef09-449b-a186-9ed0164b1986 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:12.467635 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd984d-ef09-449b-a186-9ed0164b1986 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:12.513750 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6a4b0-620d-4e89-b6cf-f746619d85de None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:12.514439 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6a4b0-620d-4e89-b6cf-f746619d85de None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:12.514910 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6a4b0-620d-4e89-b6cf-f746619d85de None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:12.517710 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 207/420] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:12.519637 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7218d705-1395-47d5-8676-cd9f8e0edf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:12.520251 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7218d705-1395-47d5-8676-cd9f8e0edf8d None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:12.520600 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7218d705-1395-47d5-8676-cd9f8e0edf8d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:12.520880 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7218d705-1395-47d5-8676-cd9f8e0edf8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:12.523497 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9faa9-b00e-49b5-8297-3453894bd30b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:12.523968 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9faa9-b00e-49b5-8297-3453894bd30b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:12.524529 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9faa9-b00e-49b5-8297-3453894bd30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:12.556276 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-43a9faa9-b00e-49b5-8297-3453894bd30b None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:12.559251 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-43a9faa9-b00e-49b5-8297-3453894bd30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:12.621722 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:12.654466 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7218d705-1395-47d5-8676-cd9f8e0edf8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:12.655960 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7218d705-1395-47d5-8676-cd9f8e0edf8d None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:12.689941 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 208/421] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:12.822765 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 214/422] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:48:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:48:12.828537 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9faa9-b00e-49b5-8297-3453894bd30b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:12.907111 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:12.969909 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9faa9-b00e-49b5-8297-3453894bd30b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:13.007940 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09e4d0-b356-4b21-9113-02d5bb18a166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:13.008600 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09e4d0-b356-4b21-9113-02d5bb18a166 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:13.008884 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09e4d0-b356-4b21-9113-02d5bb18a166 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:13.009162 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09e4d0-b356-4b21-9113-02d5bb18a166 None admin] PATH_INFO: `/v3/projects/c8fe5139a2a24ec780814c7f39d5b437` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:13.010275 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f09e4d0-b356-4b21-9113-02d5bb18a166 None admin] RBAC: Authorizing `identity:get_project(project_id=c8fe5139a2a24ec780814c7f39d5b437)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:13.011705 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f09e4d0-b356-4b21-9113-02d5bb18a166 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:13.013664 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 209/423] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:48:12 2022] GET /identity/v3/projects/c8fe5139a2a24ec780814c7f39d5b437 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:13.019077 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7218d705-1395-47d5-8676-cd9f8e0edf8d None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:13.065455 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09e4d0-b356-4b21-9113-02d5bb18a166 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:13.166165 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:13.180142 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7218d705-1395-47d5-8676-cd9f8e0edf8d None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:13.220239 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f66bc6a-00a6-434d-8e16-a10d245523ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:13.221381 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66bc6a-00a6-434d-8e16-a10d245523ac None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:13.223417 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66bc6a-00a6-434d-8e16-a10d245523ac None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:13.223886 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66bc6a-00a6-434d-8e16-a10d245523ac None admin] PATH_INFO: `/v3/groups/664a63a166be4fd9a4e70be79f046c5f` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:13.226215 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f66bc6a-00a6-434d-8e16-a10d245523ac None admin] RBAC: Authorizing `identity:get_group(group_id=664a63a166be4fd9a4e70be79f046c5f)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:13.227889 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f66bc6a-00a6-434d-8e16-a10d245523ac None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:13.230360 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 215/424] 10.222.0.27 () {56 vars in 1180 bytes} [Sun Sep 4 00:48:13 2022] GET /identity/v3/groups/664a63a166be4fd9a4e70be79f046c5f => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:13.237086 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09e4d0-b356-4b21-9113-02d5bb18a166 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:13.240489 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f66bc6a-00a6-434d-8e16-a10d245523ac None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:13.264204 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:13.298547 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf36546-5c0e-4ef1-9841-3666d6c64d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:13.299187 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf36546-5c0e-4ef1-9841-3666d6c64d97 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:13.299469 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf36546-5c0e-4ef1-9841-3666d6c64d97 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:13.299754 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf36546-5c0e-4ef1-9841-3666d6c64d97 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:13.325122 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf36546-5c0e-4ef1-9841-3666d6c64d97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:13.332332 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-bcf36546-5c0e-4ef1-9841-3666d6c64d97 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:48:13.332332 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:13.332332 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:13.332332 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:13.332332 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:13.332332 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:13.332332 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:13.332332 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:13.332332 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:13.332332 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:13.332332 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:13.332332 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:13.332332 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:13.332332 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:13.332332 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:13.332332 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:13.333760 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:13.333760 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:13.333760 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:13.333760 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:13.333760 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:13.333760 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:13.333760 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:13.333760 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:13.333760 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:13.333760 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:13.333760 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:13.333760 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:13.333760 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:13.333760 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:13.333760 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:13.333760 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:13.334849 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:13.334849 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:13.334849 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:13.334849 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:13.334849 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:13.334849 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:13.334849 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:13.334849 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:13.334849 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:13.334849 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:13.334849 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:48:13.334849 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:13.334849 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 210/425] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:48:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:13.340049 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf36546-5c0e-4ef1-9841-3666d6c64d97 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:13.462402 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f66bc6a-00a6-434d-8e16-a10d245523ac None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:13.473947 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf36546-5c0e-4ef1-9841-3666d6c64d97 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:13.508577 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c729ebf-e876-41f7-8717-a978a49fc5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:13.509622 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c729ebf-e876-41f7-8717-a978a49fc5eb None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:13.510135 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c729ebf-e876-41f7-8717-a978a49fc5eb None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:13.510635 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c729ebf-e876-41f7-8717-a978a49fc5eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:13.511666 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c729ebf-e876-41f7-8717-a978a49fc5eb None admin] RBAC: Adding query filter params (group.id=664a63a166be4fd9a4e70be79f046c5f, role.id=7df9ebc56a2a4214b2b64a5941812ef0, scope.project.id=c8fe5139a2a24ec780814c7f39d5b437) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:13.512274 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c729ebf-e876-41f7-8717-a978a49fc5eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:13.513959 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c729ebf-e876-41f7-8717-a978a49fc5eb None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:13.521178 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1340435-5b72-46d3-9764-2acce4255367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:13.521857 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1340435-5b72-46d3-9764-2acce4255367 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:13.522064 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1340435-5b72-46d3-9764-2acce4255367 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:13.522453 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1340435-5b72-46d3-9764-2acce4255367 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:13.522681 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 216/426] 10.222.0.27 () {56 vars in 1376 bytes} [Sun Sep 4 00:48:13 2022] GET /identity/v3/role_assignments?group.id=664a63a166be4fd9a4e70be79f046c5f&scope.project.id=c8fe5139a2a24ec780814c7f39d5b437&role.id=7df9ebc56a2a4214b2b64a5941812ef0 => generated 611 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:13.524105 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebaf93-c2a6-49dc-8e9a-39395b8274fa None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:13.524628 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebaf93-c2a6-49dc-8e9a-39395b8274fa None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:13.525375 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebaf93-c2a6-49dc-8e9a-39395b8274fa None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:13.527190 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 217/427] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:13.528025 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1340435-5b72-46d3-9764-2acce4255367 None admin] RBAC: Adding query filter params (name=member) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:13.528352 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1340435-5b72-46d3-9764-2acce4255367 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:13.529786 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1340435-5b72-46d3-9764-2acce4255367 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:13.546515 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e60a8-3904-4537-a3b5-44287753da87 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:13.547145 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e60a8-3904-4537-a3b5-44287753da87 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:13.547694 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e60a8-3904-4537-a3b5-44287753da87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:13.550690 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 211/428] 10.222.0.27 () {56 vars in 1101 bytes} [Sun Sep 4 00:48:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:13.555558 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1340435-5b72-46d3-9764-2acce4255367 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:13.570378 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-ea8e60a8-3904-4537-a3b5-44287753da87 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:13.572027 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ea8e60a8-3904-4537-a3b5-44287753da87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:13.593274 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:13.636292 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:13.665327 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 218/429] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 00:48:13.745249 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1340435-5b72-46d3-9764-2acce4255367 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:13.780811 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e6a4d-8b43-423b-b06b-2687d5783c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:13.781642 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e6a4d-8b43-423b-b06b-2687d5783c89 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:13.782078 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e6a4d-8b43-423b-b06b-2687d5783c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:13.782502 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e6a4d-8b43-423b-b06b-2687d5783c89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:13.854381 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201e6a4d-8b43-423b-b06b-2687d5783c89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:13.856127 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201e6a4d-8b43-423b-b06b-2687d5783c89 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:13.892847 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:13.968500 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 212/430] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:48:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 04 00:48:13.975722 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8e60a8-3904-4537-a3b5-44287753da87 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:13.986343 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e6a4d-8b43-423b-b06b-2687d5783c89 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:14.106794 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8e60a8-3904-4537-a3b5-44287753da87 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:14.112628 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e6a4d-8b43-423b-b06b-2687d5783c89 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:14.140580 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c439873b-5b26-4d8d-b289-47efc233d85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:14.141213 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439873b-5b26-4d8d-b289-47efc233d85d None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:14.141492 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439873b-5b26-4d8d-b289-47efc233d85d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:14.141763 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439873b-5b26-4d8d-b289-47efc233d85d None admin] PATH_INFO: `/v3/projects/c8fe5139a2a24ec780814c7f39d5b437` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:14.142868 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c439873b-5b26-4d8d-b289-47efc233d85d None admin] RBAC: Authorizing `identity:get_project(project_id=c8fe5139a2a24ec780814c7f39d5b437)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:14.145024 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c439873b-5b26-4d8d-b289-47efc233d85d None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:14.151003 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 219/431] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:48:13 2022] GET /identity/v3/projects/c8fe5139a2a24ec780814c7f39d5b437 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:14.151003 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237f76a-9edf-4425-8f38-95c26a3ff144 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:14.151003 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237f76a-9edf-4425-8f38-95c26a3ff144 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:14.151003 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237f76a-9edf-4425-8f38-95c26a3ff144 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:14.151003 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 220/432] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:14.153191 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2237f76a-9edf-4425-8f38-95c26a3ff144 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:14.155073 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a34b291-69cb-437b-ad04-f3cbf2393f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:14.155710 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34b291-69cb-437b-ad04-f3cbf2393f91 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:14.155989 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34b291-69cb-437b-ad04-f3cbf2393f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:14.156311 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34b291-69cb-437b-ad04-f3cbf2393f91 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:14.158302 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:14.169933 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a34b291-69cb-437b-ad04-f3cbf2393f91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:14.174046 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:14.174711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-8a34b291-69cb-437b-ad04-f3cbf2393f91 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:48:14.174711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:14.174711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:14.174711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:14.174711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:14.174711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:14.174711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:14.174711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:14.174711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:14.174711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:14.174711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:14.174711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:14.174711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:14.174711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:14.174711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:14.174711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:14.176073 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:14.176073 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:14.176073 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:14.176073 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:14.176073 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:14.176073 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:14.176073 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:14.176073 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:14.176073 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:14.176073 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:14.176073 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:14.176073 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:14.176073 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:14.176073 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:14.176073 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:14.176073 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:14.180976 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:14.180976 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:14.180976 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:14.180976 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:14.180976 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:14.180976 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:14.180976 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:14.180976 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:14.180976 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:14.180976 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:14.180976 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 04 00:48:14.180976 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:14.180976 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 213/433] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:48:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:48:14.180976 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68e357-02a8-4ca4-8f2f-29335bfdfefc None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:14.193202 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68e357-02a8-4ca4-8f2f-29335bfdfefc None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:14.193202 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68e357-02a8-4ca4-8f2f-29335bfdfefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:14.198325 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-be68e357-02a8-4ca4-8f2f-29335bfdfefc None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:14.199508 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-be68e357-02a8-4ca4-8f2f-29335bfdfefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:14.275143 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2237f76a-9edf-4425-8f38-95c26a3ff144 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:14.278029 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 214/434] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:14.278506 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68e357-02a8-4ca4-8f2f-29335bfdfefc None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:14.329723 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010cc520-78dc-4769-b792-26fd1e0c25c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:14.331006 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010cc520-78dc-4769-b792-26fd1e0c25c0 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:14.332596 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010cc520-78dc-4769-b792-26fd1e0c25c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:14.333355 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010cc520-78dc-4769-b792-26fd1e0c25c0 None admin] PATH_INFO: `/v3/groups/664a63a166be4fd9a4e70be79f046c5f` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:14.334785 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010cc520-78dc-4769-b792-26fd1e0c25c0 None admin] RBAC: Authorizing `identity:get_group(group_id=664a63a166be4fd9a4e70be79f046c5f)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:14.337334 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010cc520-78dc-4769-b792-26fd1e0c25c0 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:14.340596 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 221/435] 10.222.0.27 () {56 vars in 1180 bytes} [Sun Sep 4 00:48:14 2022] GET /identity/v3/groups/664a63a166be4fd9a4e70be79f046c5f => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:48:14.346351 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010cc520-78dc-4769-b792-26fd1e0c25c0 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:14.394794 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68e357-02a8-4ca4-8f2f-29335bfdfefc None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:14.420872 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:14.436619 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d445d8-7b93-45c0-9acf-565f0a406086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:14.437704 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d445d8-7b93-45c0-9acf-565f0a406086 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:14.438150 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d445d8-7b93-45c0-9acf-565f0a406086 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:14.439488 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d445d8-7b93-45c0-9acf-565f0a406086 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:14.440309 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d445d8-7b93-45c0-9acf-565f0a406086 None admin] RBAC: Adding query filter params (name=member) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:14.440901 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d445d8-7b93-45c0-9acf-565f0a406086 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:14.442482 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d445d8-7b93-45c0-9acf-565f0a406086 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:14.453976 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 215/436] 10.222.0.27 () {56 vars in 1101 bytes} [Sun Sep 4 00:48:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:14.459620 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:14.460140 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d445d8-7b93-45c0-9acf-565f0a406086 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:14.481135 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010cc520-78dc-4769-b792-26fd1e0c25c0 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:14.524985 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9cdb67-507f-4d17-8a8f-79f10a2e4343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:14.525597 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cdb67-507f-4d17-8a8f-79f10a2e4343 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:14.525881 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cdb67-507f-4d17-8a8f-79f10a2e4343 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:14.526159 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cdb67-507f-4d17-8a8f-79f10a2e4343 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:14.526887 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9cdb67-507f-4d17-8a8f-79f10a2e4343 None admin] RBAC: Adding query filter params (group.id=664a63a166be4fd9a4e70be79f046c5f, role.id=d1261c99e1d64f9da63adfff739561e8, scope.project.id=c8fe5139a2a24ec780814c7f39d5b437) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:14.527203 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9cdb67-507f-4d17-8a8f-79f10a2e4343 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:14.528663 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9cdb67-507f-4d17-8a8f-79f10a2e4343 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:14.535296 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 222/437] 10.222.0.27 () {56 vars in 1376 bytes} [Sun Sep 4 00:48:14 2022] GET /identity/v3/role_assignments?group.id=664a63a166be4fd9a4e70be79f046c5f&scope.project.id=c8fe5139a2a24ec780814c7f39d5b437&role.id=d1261c99e1d64f9da63adfff739561e8 => generated 611 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:48:14.535296 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80657a2b-69c2-4229-81f9-d2f28697376c None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:14.535296 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80657a2b-69c2-4229-81f9-d2f28697376c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:14.535296 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80657a2b-69c2-4229-81f9-d2f28697376c None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:14.535296 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 223/438] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:14.544273 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5104abcd-87da-47f1-9cad-eb6b0b135112 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:14.544604 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5104abcd-87da-47f1-9cad-eb6b0b135112 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:14.544865 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5104abcd-87da-47f1-9cad-eb6b0b135112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:14.563854 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-5104abcd-87da-47f1-9cad-eb6b0b135112 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:14.565142 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5104abcd-87da-47f1-9cad-eb6b0b135112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:14.583769 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d445d8-7b93-45c0-9acf-565f0a406086 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:14.623069 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7becda-f6b5-4120-b6d5-a341c164153c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:14.623685 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7becda-f6b5-4120-b6d5-a341c164153c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:14.623968 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7becda-f6b5-4120-b6d5-a341c164153c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:14.624419 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7becda-f6b5-4120-b6d5-a341c164153c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:14.642146 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 224/439] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 00:48:14.642933 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5104abcd-87da-47f1-9cad-eb6b0b135112 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:14.702974 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:14.718291 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7becda-f6b5-4120-b6d5-a341c164153c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:14.720044 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7becda-f6b5-4120-b6d5-a341c164153c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:14.744515 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:14.772550 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5104abcd-87da-47f1-9cad-eb6b0b135112 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:14.821995 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1164653f-a62b-4792-a9a2-e029b134620c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:14.822692 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164653f-a62b-4792-a9a2-e029b134620c None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:14.822972 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164653f-a62b-4792-a9a2-e029b134620c None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:14.823244 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164653f-a62b-4792-a9a2-e029b134620c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:14.834937 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1164653f-a62b-4792-a9a2-e029b134620c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:14.838488 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 216/440] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:48:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 04 00:48:14.839741 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-1164653f-a62b-4792-a9a2-e029b134620c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 00:48:14.839741 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:14.839741 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:14.839741 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:14.839741 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:14.839741 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:14.839741 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:14.839741 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:14.839741 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:14.839741 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:14.839741 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:14.839741 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:14.839741 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:14.839741 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:14.839741 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:14.839741 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:14.840694 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:14.840694 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:14.840694 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:14.840694 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:14.840694 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:14.840694 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:14.840694 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:14.840694 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:14.840694 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:14.840694 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:14.840694 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:14.840694 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:14.840694 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:14.840694 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:14.840694 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:14.840694 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:14.841489 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:14.841489 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:14.841489 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:14.841489 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:14.841489 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:14.841489 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:14.841489 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:14.841489 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:14.841489 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:14.841489 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:14.841489 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 00:48:14.841489 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:14.842099 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 225/441] 10.222.0.27 () {56 vars in 1114 bytes} [Sun Sep 4 00:48:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:14.844549 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7becda-f6b5-4120-b6d5-a341c164153c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:14.849402 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1164653f-a62b-4792-a9a2-e029b134620c None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:14.962549 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:14.971264 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1164653f-a62b-4792-a9a2-e029b134620c None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:14.983075 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7becda-f6b5-4120-b6d5-a341c164153c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:15.012614 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30769d3-b2cd-4051-9f6c-3977da3c1510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:15.013231 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30769d3-b2cd-4051-9f6c-3977da3c1510 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:15.013517 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30769d3-b2cd-4051-9f6c-3977da3c1510 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:15.013789 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30769d3-b2cd-4051-9f6c-3977da3c1510 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:15.014460 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30769d3-b2cd-4051-9f6c-3977da3c1510 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:15.014759 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30769d3-b2cd-4051-9f6c-3977da3c1510 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:15.016158 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30769d3-b2cd-4051-9f6c-3977da3c1510 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:15.025866 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 226/442] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:48:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:15.026783 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30769d3-b2cd-4051-9f6c-3977da3c1510 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:15.029492 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:15.032388 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf3616-3a37-495e-b848-cea4cf24354f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:15.032937 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf3616-3a37-495e-b848-cea4cf24354f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:15.033055 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf3616-3a37-495e-b848-cea4cf24354f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:15.033334 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf3616-3a37-495e-b848-cea4cf24354f None admin] PATH_INFO: `/v3/projects/5b12c253c6eb48c9a549a9a8436eced1` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:15.034587 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf3616-3a37-495e-b848-cea4cf24354f None admin] RBAC: Authorizing `identity:get_project(project_id=5b12c253c6eb48c9a549a9a8436eced1)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:15.036092 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf3616-3a37-495e-b848-cea4cf24354f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:15.037781 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 217/443] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:48:14 2022] GET /identity/v3/projects/5b12c253c6eb48c9a549a9a8436eced1 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:15.038551 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf3616-3a37-495e-b848-cea4cf24354f None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:15.151025 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30769d3-b2cd-4051-9f6c-3977da3c1510 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:15.158448 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf3616-3a37-495e-b848-cea4cf24354f None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:15.186267 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d39050-14d0-454d-8eff-38a7972f877e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:15.186890 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d39050-14d0-454d-8eff-38a7972f877e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:15.187169 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d39050-14d0-454d-8eff-38a7972f877e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:15.187441 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d39050-14d0-454d-8eff-38a7972f877e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:15.192038 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8788351d-8777-4e75-95dd-846fa66c028d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:15.192659 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8788351d-8777-4e75-95dd-846fa66c028d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:15.192935 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8788351d-8777-4e75-95dd-846fa66c028d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:15.193206 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8788351d-8777-4e75-95dd-846fa66c028d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:15.197672 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d39050-14d0-454d-8eff-38a7972f877e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:15.202919 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-73d39050-14d0-454d-8eff-38a7972f877e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:48:15.202919 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:15.202919 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:15.202919 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:15.202919 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:15.202919 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:15.202919 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:15.202919 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:15.202919 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:15.202919 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:15.202919 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:15.202919 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:15.202919 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:15.202919 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:15.202919 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:15.202919 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:15.203925 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:15.203925 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:15.203925 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:15.203925 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:15.203925 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:15.203925 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:15.203925 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:15.203925 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:15.203925 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:15.203925 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:15.203925 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:15.203925 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:15.203925 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:15.203925 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:15.203925 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:15.203925 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:15.204742 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:15.204742 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:15.204742 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:15.204742 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:15.204742 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:15.204742 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:15.204742 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:15.204742 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:15.204742 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:15.204742 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:15.204742 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:48:15.204742 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:15.204742 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 227/444] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:48:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:15.205901 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d39050-14d0-454d-8eff-38a7972f877e None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:15.221163 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:15.265915 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:15.268883 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8788351d-8777-4e75-95dd-846fa66c028d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:15.270361 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8788351d-8777-4e75-95dd-846fa66c028d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:15.306658 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d39050-14d0-454d-8eff-38a7972f877e None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:15.339653 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480c1cd8-e917-494f-84b2-0d8f30a43a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:15.340278 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c1cd8-e917-494f-84b2-0d8f30a43a9b None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:15.340586 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c1cd8-e917-494f-84b2-0d8f30a43a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:15.340864 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c1cd8-e917-494f-84b2-0d8f30a43a9b None admin] PATH_INFO: `/v3/groups/664a63a166be4fd9a4e70be79f046c5f` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:15.341974 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480c1cd8-e917-494f-84b2-0d8f30a43a9b None admin] RBAC: Authorizing `identity:get_group(group_id=664a63a166be4fd9a4e70be79f046c5f)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:15.343438 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480c1cd8-e917-494f-84b2-0d8f30a43a9b None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:15.345025 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 228/445] 10.222.0.27 () {56 vars in 1180 bytes} [Sun Sep 4 00:48:15 2022] GET /identity/v3/groups/664a63a166be4fd9a4e70be79f046c5f => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:15.345782 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480c1cd8-e917-494f-84b2-0d8f30a43a9b None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:15.374803 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 218/446] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:48:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:48:15.375641 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8788351d-8777-4e75-95dd-846fa66c028d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:15.432697 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:15.446835 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480c1cd8-e917-494f-84b2-0d8f30a43a9b None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:15.475350 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:15.478640 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8788351d-8777-4e75-95dd-846fa66c028d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:15.480888 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6e6b9-0c86-4fd0-9fee-b83487e2d4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:15.481495 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6e6b9-0c86-4fd0-9fee-b83487e2d4e5 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:15.481891 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6e6b9-0c86-4fd0-9fee-b83487e2d4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:15.482039 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6e6b9-0c86-4fd0-9fee-b83487e2d4e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:15.482700 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb6e6b9-0c86-4fd0-9fee-b83487e2d4e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:15.483004 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb6e6b9-0c86-4fd0-9fee-b83487e2d4e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:15.484383 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb6e6b9-0c86-4fd0-9fee-b83487e2d4e5 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:15.494108 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 229/447] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:48:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:15.494598 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6e6b9-0c86-4fd0-9fee-b83487e2d4e5 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:15.516288 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8829fc48-cba4-41d8-a9b3-fb9d8469a648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:15.516908 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829fc48-cba4-41d8-a9b3-fb9d8469a648 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:15.517187 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829fc48-cba4-41d8-a9b3-fb9d8469a648 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:15.517452 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829fc48-cba4-41d8-a9b3-fb9d8469a648 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:15.518207 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8829fc48-cba4-41d8-a9b3-fb9d8469a648 None admin] RBAC: Adding query filter params (group.id=664a63a166be4fd9a4e70be79f046c5f, role.id=d1261c99e1d64f9da63adfff739561e8, scope.project.id=5b12c253c6eb48c9a549a9a8436eced1) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:15.519038 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8829fc48-cba4-41d8-a9b3-fb9d8469a648 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:15.520442 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8829fc48-cba4-41d8-a9b3-fb9d8469a648 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:15.526121 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 219/448] 10.222.0.27 () {56 vars in 1376 bytes} [Sun Sep 4 00:48:15 2022] GET /identity/v3/role_assignments?group.id=664a63a166be4fd9a4e70be79f046c5f&scope.project.id=5b12c253c6eb48c9a549a9a8436eced1&role.id=d1261c99e1d64f9da63adfff739561e8 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:15.526807 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8829fc48-cba4-41d8-a9b3-fb9d8469a648 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:15.609888 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6e6b9-0c86-4fd0-9fee-b83487e2d4e5 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:15.639449 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8829fc48-cba4-41d8-a9b3-fb9d8469a648 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:15.649827 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec79b2-72b0-4bfc-bc01-15ee3ee416e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:15.650592 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec79b2-72b0-4bfc-bc01-15ee3ee416e4 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:15.650875 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec79b2-72b0-4bfc-bc01-15ee3ee416e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:15.651149 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec79b2-72b0-4bfc-bc01-15ee3ee416e4 None admin] PATH_INFO: `/v3/projects/5b12c253c6eb48c9a549a9a8436eced1` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:15.652424 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec79b2-72b0-4bfc-bc01-15ee3ee416e4 None admin] RBAC: Authorizing `identity:get_project(project_id=5b12c253c6eb48c9a549a9a8436eced1)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:15.653908 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec79b2-72b0-4bfc-bc01-15ee3ee416e4 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:15.660994 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 230/449] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:48:15 2022] GET /identity/v3/projects/5b12c253c6eb48c9a549a9a8436eced1 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:15.661196 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:15.663181 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec79b2-72b0-4bfc-bc01-15ee3ee416e4 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:15.678792 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b3697-15f2-4f5e-9cce-a6fb47aad173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:15.679403 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b3697-15f2-4f5e-9cce-a6fb47aad173 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:15.679687 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b3697-15f2-4f5e-9cce-a6fb47aad173 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:15.679965 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b3697-15f2-4f5e-9cce-a6fb47aad173 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:15.732114 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:15.756361 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b3697-15f2-4f5e-9cce-a6fb47aad173 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:15.757885 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b3697-15f2-4f5e-9cce-a6fb47aad173 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:15.780565 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec79b2-72b0-4bfc-bc01-15ee3ee416e4 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:15.824124 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe4bbc3-fa79-4981-8981-58df994a1594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:15.824803 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4bbc3-fa79-4981-8981-58df994a1594 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:15.825089 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4bbc3-fa79-4981-8981-58df994a1594 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:15.825370 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4bbc3-fa79-4981-8981-58df994a1594 None admin] PATH_INFO: `/v3/groups/664a63a166be4fd9a4e70be79f046c5f` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:15.826521 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe4bbc3-fa79-4981-8981-58df994a1594 None admin] RBAC: Authorizing `identity:get_group(group_id=664a63a166be4fd9a4e70be79f046c5f)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:15.828083 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe4bbc3-fa79-4981-8981-58df994a1594 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:15.832696 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 231/450] 10.222.0.27 () {56 vars in 1180 bytes} [Sun Sep 4 00:48:15 2022] GET /identity/v3/groups/664a63a166be4fd9a4e70be79f046c5f => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:48:15.836419 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe4bbc3-fa79-4981-8981-58df994a1594 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:15.882805 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 220/451] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:48:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:48:15.889021 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b3697-15f2-4f5e-9cce-a6fb47aad173 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:15.909465 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:15.946029 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe4bbc3-fa79-4981-8981-58df994a1594 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:15.972852 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:15.980652 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72200c02-d22d-4f7e-8de1-5cb31ba2ddd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:15.981247 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72200c02-d22d-4f7e-8de1-5cb31ba2ddd3 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:15.981557 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72200c02-d22d-4f7e-8de1-5cb31ba2ddd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:15.981863 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72200c02-d22d-4f7e-8de1-5cb31ba2ddd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:15.982631 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72200c02-d22d-4f7e-8de1-5cb31ba2ddd3 None admin] RBAC: Adding query filter params (group.id=664a63a166be4fd9a4e70be79f046c5f, role.id=7df9ebc56a2a4214b2b64a5941812ef0, scope.project.id=5b12c253c6eb48c9a549a9a8436eced1) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:15.982888 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72200c02-d22d-4f7e-8de1-5cb31ba2ddd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:15.984316 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72200c02-d22d-4f7e-8de1-5cb31ba2ddd3 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:15.990118 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 232/452] 10.222.0.27 () {56 vars in 1376 bytes} [Sun Sep 4 00:48:15 2022] GET /identity/v3/role_assignments?group.id=664a63a166be4fd9a4e70be79f046c5f&scope.project.id=5b12c253c6eb48c9a549a9a8436eced1&role.id=7df9ebc56a2a4214b2b64a5941812ef0 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:48:15.994993 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b3697-15f2-4f5e-9cce-a6fb47aad173 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:16.031713 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11b07b-29a5-4c30-abab-4382d490c8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:16.032355 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11b07b-29a5-4c30-abab-4382d490c8fd None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:16.032669 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11b07b-29a5-4c30-abab-4382d490c8fd None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:16.032939 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11b07b-29a5-4c30-abab-4382d490c8fd None admin] PATH_INFO: `/v3/projects/a1d8c40dd6c04e8e8fe6b0c2b34723cd` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:16.034202 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b11b07b-29a5-4c30-abab-4382d490c8fd None admin] RBAC: Authorizing `identity:get_project(project_id=a1d8c40dd6c04e8e8fe6b0c2b34723cd)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:16.035676 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b11b07b-29a5-4c30-abab-4382d490c8fd None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:16.037428 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 221/453] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:48:15 2022] GET /identity/v3/projects/a1d8c40dd6c04e8e8fe6b0c2b34723cd => generated 344 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:48:16.043720 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72200c02-d22d-4f7e-8de1-5cb31ba2ddd3 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:16.149240 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72200c02-d22d-4f7e-8de1-5cb31ba2ddd3 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:16.184332 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b229da-3928-4422-9be8-696911ebb5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:16.185007 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b229da-3928-4422-9be8-696911ebb5d1 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:16.185294 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b229da-3928-4422-9be8-696911ebb5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:16.185573 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b229da-3928-4422-9be8-696911ebb5d1 None admin] PATH_INFO: `/v3/groups/fcbc87e1a8a24d9abda6502eefa59888` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:16.186697 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b229da-3928-4422-9be8-696911ebb5d1 None admin] RBAC: Authorizing `identity:get_group(group_id=fcbc87e1a8a24d9abda6502eefa59888)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:16.188148 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b229da-3928-4422-9be8-696911ebb5d1 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:16.189756 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 233/454] 10.222.0.27 () {56 vars in 1180 bytes} [Sun Sep 4 00:48:16 2022] GET /identity/v3/groups/fcbc87e1a8a24d9abda6502eefa59888 => generated 229 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:16.195952 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11b07b-29a5-4c30-abab-4382d490c8fd None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:16.289679 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:16.308188 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11b07b-29a5-4c30-abab-4382d490c8fd None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:16.343564 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ddee7-453b-4ee1-a4da-099c5026f0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:16.344162 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ddee7-453b-4ee1-a4da-099c5026f0c9 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:16.344442 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ddee7-453b-4ee1-a4da-099c5026f0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:16.344761 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ddee7-453b-4ee1-a4da-099c5026f0c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:16.345475 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8ddee7-453b-4ee1-a4da-099c5026f0c9 None admin] RBAC: Adding query filter params (group.id=fcbc87e1a8a24d9abda6502eefa59888, role.id=c8a59d34ee2f4cefa6940cac778260c7, scope.project.id=a1d8c40dd6c04e8e8fe6b0c2b34723cd) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:16.345781 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8ddee7-453b-4ee1-a4da-099c5026f0c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:16.347581 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8ddee7-453b-4ee1-a4da-099c5026f0c9 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:16.353204 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 222/455] 10.222.0.27 () {56 vars in 1376 bytes} [Sun Sep 4 00:48:16 2022] GET /identity/v3/role_assignments?group.id=fcbc87e1a8a24d9abda6502eefa59888&scope.project.id=a1d8c40dd6c04e8e8fe6b0c2b34723cd&role.id=c8a59d34ee2f4cefa6940cac778260c7 => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:18.636907 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e461d5-ee91-4470-bcc5-d9f1c4753399 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:18.637937 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e461d5-ee91-4470-bcc5-d9f1c4753399 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:18.638469 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e461d5-ee91-4470-bcc5-d9f1c4753399 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:18.640411 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 234/456] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:18.647005 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea14f97-dcba-4ed0-943c-81c4cbcda117 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:18.647897 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea14f97-dcba-4ed0-943c-81c4cbcda117 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:18.648381 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea14f97-dcba-4ed0-943c-81c4cbcda117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:18.673850 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1ea14f97-dcba-4ed0-943c-81c4cbcda117 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:18.675313 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1ea14f97-dcba-4ed0-943c-81c4cbcda117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:18.763814 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 223/457] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:18.779122 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e1a12-276b-4497-907a-f53860e9a20c None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:18.779405 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e1a12-276b-4497-907a-f53860e9a20c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:18.779682 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e1a12-276b-4497-907a-f53860e9a20c None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:18.781241 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 235/458] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:18.787083 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8257951-1862-4ce3-85dc-09cb7c34e295 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:18.787375 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8257951-1862-4ce3-85dc-09cb7c34e295 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:18.787659 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8257951-1862-4ce3-85dc-09cb7c34e295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:18.813495 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-e8257951-1862-4ce3-85dc-09cb7c34e295 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:18.814699 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e8257951-1862-4ce3-85dc-09cb7c34e295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:18.901177 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 224/459] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:19.068543 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30f704-766c-4533-aeb7-404b7bd78f98 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:19.069210 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30f704-766c-4533-aeb7-404b7bd78f98 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:19.069705 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30f704-766c-4533-aeb7-404b7bd78f98 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:19.071545 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 236/460] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:19.076535 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2613a-15e0-4c25-8480-8f20914c02e3 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:19.076834 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2613a-15e0-4c25-8480-8f20914c02e3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:19.077118 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2613a-15e0-4c25-8480-8f20914c02e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:19.105434 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-72f2613a-15e0-4c25-8480-8f20914c02e3 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:19.106642 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-72f2613a-15e0-4c25-8480-8f20914c02e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:19.109597 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:19.199126 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 225/461] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 00:48:19.239464 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30f704-766c-4533-aeb7-404b7bd78f98 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:19.263453 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2613a-15e0-4c25-8480-8f20914c02e3 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:19.331842 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:19.379650 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2613a-15e0-4c25-8480-8f20914c02e3 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:19.416186 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaf6a95-bc8e-45aa-b69d-5c3b6118973a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:19.417170 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf6a95-bc8e-45aa-b69d-5c3b6118973a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:19.417642 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf6a95-bc8e-45aa-b69d-5c3b6118973a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:19.418124 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf6a95-bc8e-45aa-b69d-5c3b6118973a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:19.426137 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30f704-766c-4533-aeb7-404b7bd78f98 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:19.427360 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:19.430364 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaf6a95-bc8e-45aa-b69d-5c3b6118973a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:19.431995 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaf6a95-bc8e-45aa-b69d-5c3b6118973a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:19.447047 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-2aaf6a95-bc8e-45aa-b69d-5c3b6118973a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:19.447047 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:19.447047 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:19.447047 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:19.447047 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:19.447047 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:19.447047 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:19.447047 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:19.447047 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:19.447047 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:19.447047 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:19.447047 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:19.447047 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:19.447047 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:19.447047 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:19.447047 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:19.447992 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:19.447992 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:19.447992 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:19.447992 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:19.447992 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:19.447992 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:19.447992 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:19.447992 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:19.447992 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:19.447992 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:19.447992 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:19.447992 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:19.447992 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:19.447992 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:19.447992 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:19.447992 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:19.447992 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:19.448881 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:19.448881 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:19.448881 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:19.451023 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 226/462] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:48:19.458185 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaf6a95-bc8e-45aa-b69d-5c3b6118973a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:19.509260 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c1701-f066-4a6f-9c45-8a9e38814801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:19.509850 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c1701-f066-4a6f-9c45-8a9e38814801 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:19.510137 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c1701-f066-4a6f-9c45-8a9e38814801 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:19.510414 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c1701-f066-4a6f-9c45-8a9e38814801 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:19.537524 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c1701-f066-4a6f-9c45-8a9e38814801 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:19.539013 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c1701-f066-4a6f-9c45-8a9e38814801 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:19.566522 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaf6a95-bc8e-45aa-b69d-5c3b6118973a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:19.575428 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-5a5c1701-f066-4a6f-9c45-8a9e38814801 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:19.575428 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:19.575428 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:19.575428 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:19.575428 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:19.575428 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:19.575428 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:19.575428 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:19.575428 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:19.575428 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:19.575428 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:19.575428 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:19.575428 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:19.575428 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:19.575428 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:19.575428 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:19.576340 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:19.576340 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:19.576340 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:19.576340 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:19.576340 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:19.576340 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:19.576340 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:19.576340 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:19.576340 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:19.576340 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:19.576340 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:19.576340 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:19.576340 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:19.576340 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:19.576340 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:19.576340 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:19.576340 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:19.577244 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:19.577244 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:19.577244 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:19.577244 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 237/463] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:19.582963 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c1701-f066-4a6f-9c45-8a9e38814801 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:19.607300 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86987d35-4e59-40c2-9055-ba060d5064f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:19.608322 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86987d35-4e59-40c2-9055-ba060d5064f7 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:19.608820 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86987d35-4e59-40c2-9055-ba060d5064f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:19.609280 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86987d35-4e59-40c2-9055-ba060d5064f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:19.610107 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86987d35-4e59-40c2-9055-ba060d5064f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:19.610595 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86987d35-4e59-40c2-9055-ba060d5064f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:19.612241 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86987d35-4e59-40c2-9055-ba060d5064f7 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:19.625238 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 227/464] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:19.626298 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86987d35-4e59-40c2-9055-ba060d5064f7 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:19.687417 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:19.699129 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c1701-f066-4a6f-9c45-8a9e38814801 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:19.713807 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:19.738645 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dce1ab-8270-4c59-962f-e575953598f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:19.739267 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dce1ab-8270-4c59-962f-e575953598f8 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:19.739554 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dce1ab-8270-4c59-962f-e575953598f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:19.739785 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86987d35-4e59-40c2-9055-ba060d5064f7 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:19.739880 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dce1ab-8270-4c59-962f-e575953598f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:19.742023 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dce1ab-8270-4c59-962f-e575953598f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:19.742023 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dce1ab-8270-4c59-962f-e575953598f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:19.743451 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dce1ab-8270-4c59-962f-e575953598f8 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:19.769237 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 238/465] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:19.769671 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dce1ab-8270-4c59-962f-e575953598f8 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:19.777106 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcad59c-1bbb-4678-bbfb-caa236025c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:19.777735 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcad59c-1bbb-4678-bbfb-caa236025c70 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:19.778012 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcad59c-1bbb-4678-bbfb-caa236025c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:19.778281 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcad59c-1bbb-4678-bbfb-caa236025c70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:19.789252 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcad59c-1bbb-4678-bbfb-caa236025c70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:19.790933 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcad59c-1bbb-4678-bbfb-caa236025c70 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:19.805184 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-9dcad59c-1bbb-4678-bbfb-caa236025c70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:19.805184 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:19.805184 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:19.805184 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:19.805184 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:19.805184 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:19.805184 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:19.805184 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:19.805184 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:19.805184 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:19.805184 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:19.805184 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:19.805184 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:19.805184 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:19.805184 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:19.805184 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:19.806087 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:19.806087 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:19.806087 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:19.806087 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:19.806087 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:19.806087 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:19.806087 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:19.806087 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:19.806087 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:19.806087 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:19.806087 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:19.806087 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:19.806087 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:19.806087 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:19.806087 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:19.806087 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:19.806087 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:19.806949 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:19.806949 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:19.806949 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:19.806949 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 228/466] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:19.807583 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27a32c-5220-4602-9f06-5126e73255ed None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:19.807677 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27a32c-5220-4602-9f06-5126e73255ed None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:19.807942 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27a32c-5220-4602-9f06-5126e73255ed None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:19.809339 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 229/467] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:19.809853 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27a32c-5220-4602-9f06-5126e73255ed None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:19.893055 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dce1ab-8270-4c59-962f-e575953598f8 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:19.919456 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27a32c-5220-4602-9f06-5126e73255ed None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:19.922798 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:19.928589 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b063b6ef-b11d-487b-903b-5fa1b35eb330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:19.929196 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063b6ef-b11d-487b-903b-5fa1b35eb330 None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:19.929477 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063b6ef-b11d-487b-903b-5fa1b35eb330 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:19.929748 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063b6ef-b11d-487b-903b-5fa1b35eb330 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:19.930553 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b063b6ef-b11d-487b-903b-5fa1b35eb330 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:19.931988 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b063b6ef-b11d-487b-903b-5fa1b35eb330 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:19.955583 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8020862-02f8-405e-979d-3990a829de88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:19.956932 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8020862-02f8-405e-979d-3990a829de88 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:19.957260 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8020862-02f8-405e-979d-3990a829de88 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:19.957537 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8020862-02f8-405e-979d-3990a829de88 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:19.958210 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8020862-02f8-405e-979d-3990a829de88 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:19.959631 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8020862-02f8-405e-979d-3990a829de88 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:20.045606 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 239/468] 10.222.0.27 () {60 vars in 1132 bytes} [Sun Sep 4 00:48:19 2022] POST /identity/v3/users => generated 248 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:48:20.046995 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa58b0-b819-44b6-b60b-ebf0afbd5d7f None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:20.047334 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa58b0-b819-44b6-b60b-ebf0afbd5d7f None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:20.047615 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa58b0-b819-44b6-b60b-ebf0afbd5d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:20.070319 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-74fa58b0-b819-44b6-b60b-ebf0afbd5d7f None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:20.071877 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-74fa58b0-b819-44b6-b60b-ebf0afbd5d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:20.084856 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 230/469] 10.222.0.27 () {60 vars in 1132 bytes} [Sun Sep 4 00:48:19 2022] POST /identity/v3/users => generated 246 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:48:20.086239 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8020862-02f8-405e-979d-3990a829de88 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:20.105006 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:20.195498 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 240/470] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:20.197203 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210acc97-6385-461f-b4bf-954d6025699d None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:20.197671 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210acc97-6385-461f-b4bf-954d6025699d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:20.198687 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210acc97-6385-461f-b4bf-954d6025699d None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:20.206300 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 241/471] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:20 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:20.208802 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a1d59-83a9-4350-8c0f-c9cd76697e56 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:20.209243 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a1d59-83a9-4350-8c0f-c9cd76697e56 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:20.209762 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a1d59-83a9-4350-8c0f-c9cd76697e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:20.241763 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-b13a1d59-83a9-4350-8c0f-c9cd76697e56 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:20.243063 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b13a1d59-83a9-4350-8c0f-c9cd76697e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:20.282143 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8020862-02f8-405e-979d-3990a829de88 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:20.327110 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78fadb5-846e-46f8-a2f5-205cc12ba80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:20.327646 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fadb5-846e-46f8-a2f5-205cc12ba80c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:20.327929 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fadb5-846e-46f8-a2f5-205cc12ba80c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:20.328216 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fadb5-846e-46f8-a2f5-205cc12ba80c None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:20.328895 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78fadb5-846e-46f8-a2f5-205cc12ba80c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:20.329262 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78fadb5-846e-46f8-a2f5-205cc12ba80c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:20.331210 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78fadb5-846e-46f8-a2f5-205cc12ba80c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:20.334138 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 242/472] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:20.344851 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 231/473] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:20.351425 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a1d59-83a9-4350-8c0f-c9cd76697e56 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:20.445070 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:20.454782 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a1d59-83a9-4350-8c0f-c9cd76697e56 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:20.491067 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb319f3e-0ecd-4304-9fcd-d19fa4c46aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:20.491671 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb319f3e-0ecd-4304-9fcd-d19fa4c46aa1 None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:20.491949 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb319f3e-0ecd-4304-9fcd-d19fa4c46aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:20.492221 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb319f3e-0ecd-4304-9fcd-d19fa4c46aa1 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:20.492915 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb319f3e-0ecd-4304-9fcd-d19fa4c46aa1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:20.494367 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb319f3e-0ecd-4304-9fcd-d19fa4c46aa1 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:20.506171 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78fadb5-846e-46f8-a2f5-205cc12ba80c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:20.530017 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:20.602493 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 243/474] 10.222.0.27 () {60 vars in 1132 bytes} [Sun Sep 4 00:48:20 2022] POST /identity/v3/users => generated 248 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 00:48:20.628659 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78fadb5-846e-46f8-a2f5-205cc12ba80c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:20.662492 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb319f3e-0ecd-4304-9fcd-d19fa4c46aa1 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:20.688235 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4a87f8-c6da-4447-a06b-323f4d2de4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:20.688972 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a87f8-c6da-4447-a06b-323f4d2de4e9 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:20.689174 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a87f8-c6da-4447-a06b-323f4d2de4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:20.689441 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a87f8-c6da-4447-a06b-323f4d2de4e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:20.707724 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4a87f8-c6da-4447-a06b-323f4d2de4e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:20.709285 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4a87f8-c6da-4447-a06b-323f4d2de4e9 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:20.730650 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-4e4a87f8-c6da-4447-a06b-323f4d2de4e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:20.730650 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:20.730650 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:20.730650 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:20.730650 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:20.730650 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:20.730650 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:20.730650 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:20.730650 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:20.730650 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:20.730650 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:20.730650 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:20.730650 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:20.730650 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:20.730650 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:20.730650 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:20.731632 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:20.731632 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:20.731632 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:20.731632 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:20.731632 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:20.731632 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:20.731632 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:20.731632 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:20.731632 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:20.731632 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:20.731632 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:20.731632 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:20.731632 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:20.731632 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:20.731632 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:20.731632 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:20.731632 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:20.732491 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:20.732491 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:20.732491 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:20.732491 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 232/475] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:20.739968 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4a87f8-c6da-4447-a06b-323f4d2de4e9 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:20.817522 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb319f3e-0ecd-4304-9fcd-d19fa4c46aa1 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:20.862226 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:20.873103 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f2e3d-3d98-4f7d-af9a-bf02ef3f7850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:20.873725 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f2e3d-3d98-4f7d-af9a-bf02ef3f7850 None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:20.874007 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f2e3d-3d98-4f7d-af9a-bf02ef3f7850 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:20.874282 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f2e3d-3d98-4f7d-af9a-bf02ef3f7850 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:20.877694 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84f2e3d-3d98-4f7d-af9a-bf02ef3f7850 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:20.877694 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84f2e3d-3d98-4f7d-af9a-bf02ef3f7850 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:20.889654 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4a87f8-c6da-4447-a06b-323f4d2de4e9 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:20.919888 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.sql.core [None req-e84f2e3d-3d98-4f7d-af9a-bf02ef3f7850 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 04 00:48:20.919888 dv-cinder-master-3-smb devstack@keystone.service[81630]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 04 00:48:20.919888 dv-cinder-master-3-smb devstack@keystone.service[81630]: [parameters: {'id': 'f2bf3fef1bee45b39db08002e6f991d0', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 04 00:48:20.919888 dv-cinder-master-3-smb devstack@keystone.service[81630]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81630) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 04 00:48:20.931664 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468871d3-0d14-4bf0-979c-34635ea81e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:20.932351 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468871d3-0d14-4bf0-979c-34635ea81e09 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:20.932751 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468871d3-0d14-4bf0-979c-34635ea81e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:20.933096 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468871d3-0d14-4bf0-979c-34635ea81e09 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:20.933809 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468871d3-0d14-4bf0-979c-34635ea81e09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:20.934206 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468871d3-0d14-4bf0-979c-34635ea81e09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:20.935662 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468871d3-0d14-4bf0-979c-34635ea81e09 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:20.943864 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-e84f2e3d-3d98-4f7d-af9a-bf02ef3f7850 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 04 00:48:20.943864 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:20.943864 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 04 00:48:20.943864 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 04 00:48:20.943864 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 04 00:48:20.943864 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 04 00:48:20.943864 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 04 00:48:20.943864 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application result = self._query(query) Sep 04 00:48:20.943864 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 04 00:48:20.943864 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application conn.query(q) Sep 04 00:48:20.943864 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 04 00:48:20.943864 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 04 00:48:20.943864 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 04 00:48:20.943864 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application result.read() Sep 04 00:48:20.943864 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 04 00:48:20.944936 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 04 00:48:20.944936 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 04 00:48:20.944936 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 04 00:48:20.944936 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 04 00:48:20.944936 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 04 00:48:20.944936 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 04 00:48:20.944936 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 04 00:48:20.944936 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 04 00:48:20.944936 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:20.944936 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 04 00:48:20.944936 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:20.944936 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:20.944936 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 04 00:48:20.944936 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 04 00:48:20.944936 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 04 00:48:20.944936 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return ref.to_dict() Sep 04 00:48:20.944936 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 04 00:48:20.944936 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application next(self.gen) Sep 04 00:48:20.945860 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 04 00:48:20.945860 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application yield resource Sep 04 00:48:20.945860 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 04 00:48:20.945860 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application next(self.gen) Sep 04 00:48:20.945860 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 04 00:48:20.945860 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application self.session.rollback() Sep 04 00:48:20.945860 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 04 00:48:20.945860 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application self.force_reraise() Sep 04 00:48:20.945860 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 04 00:48:20.945860 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise self.value Sep 04 00:48:20.945860 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 04 00:48:20.945860 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 04 00:48:20.945860 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 04 00:48:20.945860 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application session.commit() Sep 04 00:48:20.945860 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 04 00:48:20.945860 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 04 00:48:20.945860 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 04 00:48:20.946723 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._prepare_impl() Sep 04 00:48:20.946723 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 04 00:48:20.946723 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application self.session.flush() Sep 04 00:48:20.946723 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 04 00:48:20.946723 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._flush(objects) Sep 04 00:48:20.946723 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 04 00:48:20.946723 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 04 00:48:20.946723 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 04 00:48:20.946723 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application compat.raise_( Sep 04 00:48:20.946723 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 04 00:48:20.946723 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception Sep 04 00:48:20.946723 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 04 00:48:20.946723 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application flush_context.execute() Sep 04 00:48:20.946723 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 04 00:48:20.946723 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rec.execute(self) Sep 04 00:48:20.946723 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 04 00:48:20.946723 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 04 00:48:20.947653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 04 00:48:20.947653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 04 00:48:20.947653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 04 00:48:20.947653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 04 00:48:20.947653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 04 00:48:20.947653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 04 00:48:20.947653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 04 00:48:20.947653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 04 00:48:20.947653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 04 00:48:20.947653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 04 00:48:20.947653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 04 00:48:20.947653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 04 00:48:20.947653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 04 00:48:20.947653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 04 00:48:20.947653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 04 00:48:20.947653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception Sep 04 00:48:20.948453 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 04 00:48:20.948453 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 04 00:48:20.948453 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 04 00:48:20.948453 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 04 00:48:20.948453 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 04 00:48:20.948453 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application result = self._query(query) Sep 04 00:48:20.948453 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 04 00:48:20.948453 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application conn.query(q) Sep 04 00:48:20.948453 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 04 00:48:20.948453 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 04 00:48:20.948453 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 04 00:48:20.948453 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application result.read() Sep 04 00:48:20.948453 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 04 00:48:20.948453 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 04 00:48:20.948453 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 04 00:48:20.948453 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 04 00:48:20.949965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 04 00:48:20.949965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 04 00:48:20.949965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 04 00:48:20.949965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 04 00:48:20.949965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 04 00:48:20.949965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 04 00:48:20.949965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [parameters: {'id': 'f2bf3fef1bee45b39db08002e6f991d0', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 04 00:48:20.949965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 04 00:48:20.949965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:20.949965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 00:48:20.949965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:20.949965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:20.949965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:20.949965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:20.949965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:20.949965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:20.952189 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:20.952189 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:20.952189 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:20.952189 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:20.952189 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:20.952189 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:20.952189 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 04 00:48:20.952189 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 04 00:48:20.952189 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:20.952189 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:20.952189 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 04 00:48:20.952189 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 04 00:48:20.952189 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 04 00:48:20.952189 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 04 00:48:20.952189 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 04 00:48:20.952189 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:20.960495 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 244/476] 10.222.0.27 () {60 vars in 1131 bytes} [Sun Sep 4 00:48:20 2022] POST /identity/v3/roles => generated 144 bytes in 290 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:20.960657 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 233/477] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:20.960806 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f2e3d-3d98-4f7d-af9a-bf02ef3f7850 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:20.960951 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468871d3-0d14-4bf0-979c-34635ea81e09 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:20.988837 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:21.077292 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468871d3-0d14-4bf0-979c-34635ea81e09 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:21.078971 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f2e3d-3d98-4f7d-af9a-bf02ef3f7850 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:21.109981 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:21.120395 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf108fb-0d7e-4645-bf9d-10add5d1a45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:21.121788 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf108fb-0d7e-4645-bf9d-10add5d1a45f None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:21.122552 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fabc703-ae31-4365-832f-1787b3e72b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:21.123386 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabc703-ae31-4365-832f-1787b3e72b79 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:21.123830 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf108fb-0d7e-4645-bf9d-10add5d1a45f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:21.124268 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf108fb-0d7e-4645-bf9d-10add5d1a45f None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:21.124742 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabc703-ae31-4365-832f-1787b3e72b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:21.125588 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf108fb-0d7e-4645-bf9d-10add5d1a45f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:21.126039 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabc703-ae31-4365-832f-1787b3e72b79 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:21.128950 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf108fb-0d7e-4645-bf9d-10add5d1a45f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:21.149156 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fabc703-ae31-4365-832f-1787b3e72b79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:21.155999 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-9fabc703-ae31-4365-832f-1787b3e72b79 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 00:48:21.155999 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:21.155999 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:21.155999 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:21.155999 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:21.155999 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:21.155999 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:21.155999 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:21.155999 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:21.155999 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:21.155999 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:21.155999 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:21.155999 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:21.155999 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:21.155999 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:21.155999 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:21.156922 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:21.156922 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:21.156922 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:21.156922 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:21.156922 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:21.156922 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:21.156922 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:21.156922 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:21.156922 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:21.156922 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:21.156922 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:21.156922 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:21.156922 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:21.156922 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:21.156922 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:21.156922 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:21.157702 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:21.157702 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:21.157702 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:21.157702 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:21.157702 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:21.157702 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:21.157702 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:21.157702 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:21.157702 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:21.157702 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:21.157702 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 04 00:48:21.157702 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:21.161642 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 245/478] 10.222.0.27 () {56 vars in 1114 bytes} [Sun Sep 4 00:48:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:21.167373 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fabc703-ae31-4365-832f-1787b3e72b79 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:21.233838 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 234/479] 10.222.0.27 () {60 vars in 1132 bytes} [Sun Sep 4 00:48:20 2022] POST /identity/v3/users => generated 249 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 00:48:21.269242 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:21.319644 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fabc703-ae31-4365-832f-1787b3e72b79 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:21.370851 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1302591b-c1b4-4fe7-9983-04c751fcf266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:21.371463 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302591b-c1b4-4fe7-9983-04c751fcf266 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:21.371674 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302591b-c1b4-4fe7-9983-04c751fcf266 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:21.371950 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302591b-c1b4-4fe7-9983-04c751fcf266 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:21.372670 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1302591b-c1b4-4fe7-9983-04c751fcf266 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:21.372973 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1302591b-c1b4-4fe7-9983-04c751fcf266 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:21.375231 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1302591b-c1b4-4fe7-9983-04c751fcf266 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:21.408513 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 246/480] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:48:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:48:22.561513 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc88bc9-35e1-474a-b1c4-ebc4f62eed9b None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:22.562049 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05772e7d-14e3-4451-b83a-9fe589093325 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:22.562386 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc88bc9-35e1-474a-b1c4-ebc4f62eed9b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:22.562459 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05772e7d-14e3-4451-b83a-9fe589093325 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:22.562525 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc88bc9-35e1-474a-b1c4-ebc4f62eed9b None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:22.562589 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05772e7d-14e3-4451-b83a-9fe589093325 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:22.562589 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 247/481] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:22.563204 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 235/482] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:22.566452 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245e7c4-8718-42b1-95c7-60213c7adeef None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:22.566736 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245e7c4-8718-42b1-95c7-60213c7adeef None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:22.567014 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245e7c4-8718-42b1-95c7-60213c7adeef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:22.571995 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38039b71-dbf2-4ee3-b0ce-43b00eb7e336 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:22.571995 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38039b71-dbf2-4ee3-b0ce-43b00eb7e336 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:22.571995 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38039b71-dbf2-4ee3-b0ce-43b00eb7e336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:22.589356 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-38039b71-dbf2-4ee3-b0ce-43b00eb7e336 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:22.590652 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-d245e7c4-8718-42b1-95c7-60213c7adeef None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:22.591114 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-38039b71-dbf2-4ee3-b0ce-43b00eb7e336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:22.591822 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d245e7c4-8718-42b1-95c7-60213c7adeef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:22.685330 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 236/483] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 00:48:22.709973 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 248/484] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:23.038659 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38039b71-dbf2-4ee3-b0ce-43b00eb7e336 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:23.164661 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38039b71-dbf2-4ee3-b0ce-43b00eb7e336 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:23.186410 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:23.201056 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b65625-99e4-4a13-b7d8-8b7636e90ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:23.201688 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b65625-99e4-4a13-b7d8-8b7636e90ce8 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:23.201972 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b65625-99e4-4a13-b7d8-8b7636e90ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:23.202251 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b65625-99e4-4a13-b7d8-8b7636e90ce8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:23.213520 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b65625-99e4-4a13-b7d8-8b7636e90ce8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:23.218405 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-b0b65625-99e4-4a13-b7d8-8b7636e90ce8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:23.218405 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:23.218405 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:23.218405 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:23.218405 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:23.218405 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:23.218405 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:23.218405 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:23.218405 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:23.218405 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:23.218405 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:23.218405 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:23.218405 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:23.218405 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:23.218405 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:23.218405 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:23.219257 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:23.219257 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:23.219257 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:23.219257 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:23.219257 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:23.219257 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:23.219257 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:23.219257 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:23.219257 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:23.219257 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:23.219257 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:23.219257 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:23.219257 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:23.219257 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:23.219257 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:23.219257 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:23.220219 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:23.220219 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:23.220219 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:23.220219 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:23.220219 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:23.220219 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:23.220219 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:23.220219 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:23.220219 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:23.220219 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:23.220219 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:23.220219 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:23.220219 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 237/485] 10.222.0.27 () {56 vars in 1102 bytes} [Sun Sep 4 00:48:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:23.229905 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245e7c4-8718-42b1-95c7-60213c7adeef None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:23.249707 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b65625-99e4-4a13-b7d8-8b7636e90ce8 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:23.340323 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:23.367798 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b65625-99e4-4a13-b7d8-8b7636e90ce8 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:23.395303 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245e7c4-8718-42b1-95c7-60213c7adeef None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:23.414518 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b70d2a-d8ba-442a-909b-50485d9cd30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:23.415879 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b70d2a-d8ba-442a-909b-50485d9cd30e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:23.416168 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b70d2a-d8ba-442a-909b-50485d9cd30e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:23.416445 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b70d2a-d8ba-442a-909b-50485d9cd30e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:23.431397 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b70d2a-d8ba-442a-909b-50485d9cd30e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:23.437211 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-e2b70d2a-d8ba-442a-909b-50485d9cd30e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:23.437211 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:23.437211 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:23.437211 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:23.437211 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:23.437211 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:23.437211 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:23.437211 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:23.437211 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:23.437211 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:23.437211 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:23.437211 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:23.437211 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:23.437211 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:23.437211 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:23.437211 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:23.438111 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:23.438111 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:23.438111 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:23.438111 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:23.438111 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:23.438111 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:23.438111 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:23.438111 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:23.438111 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:23.438111 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:23.438111 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:23.438111 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:23.438111 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:23.438111 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:23.438111 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:23.438111 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:23.438913 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:23.438913 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:23.438913 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:23.438913 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:23.438913 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:23.438913 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:23.438913 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:23.438913 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:23.438913 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:23.438913 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:23.438913 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:23.438913 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:23.438913 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 238/486] 10.222.0.27 () {56 vars in 1102 bytes} [Sun Sep 4 00:48:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:48:23.440690 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0a449-cbe2-4cb1-abe9-480f251c2899 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:23.440793 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0a449-cbe2-4cb1-abe9-480f251c2899 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:23.441045 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0a449-cbe2-4cb1-abe9-480f251c2899 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:23.442217 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05745209-6f28-4ca0-8bc4-2c8cc8db32ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:23.443057 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05745209-6f28-4ca0-8bc4-2c8cc8db32ed None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:23.443547 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05745209-6f28-4ca0-8bc4-2c8cc8db32ed None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:23.444013 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05745209-6f28-4ca0-8bc4-2c8cc8db32ed None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:23.445210 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05745209-6f28-4ca0-8bc4-2c8cc8db32ed None admin] RBAC: Adding query filter params (name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:23.445715 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05745209-6f28-4ca0-8bc4-2c8cc8db32ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:23.446377 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 239/487] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:23.447154 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0a449-cbe2-4cb1-abe9-480f251c2899 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:23.447834 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05745209-6f28-4ca0-8bc4-2c8cc8db32ed None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:23.458984 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 249/488] 10.222.0.27 () {56 vars in 1103 bytes} [Sun Sep 4 00:48:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:23.460662 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df2da71-7278-4043-bb45-77eca30bd181 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:23.461209 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df2da71-7278-4043-bb45-77eca30bd181 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:23.461683 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df2da71-7278-4043-bb45-77eca30bd181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:23.481009 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-5df2da71-7278-4043-bb45-77eca30bd181 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:23.482627 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5df2da71-7278-4043-bb45-77eca30bd181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:23.499841 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:23.576492 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 250/489] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:23.576492 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df2da71-7278-4043-bb45-77eca30bd181 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:23.583554 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0a449-cbe2-4cb1-abe9-480f251c2899 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:23.633530 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:23.648783 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269f97c-4fb7-482d-bd8d-f2683ab938ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:23.649568 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269f97c-4fb7-482d-bd8d-f2683ab938ad None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:23.649964 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269f97c-4fb7-482d-bd8d-f2683ab938ad None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:23.650345 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269f97c-4fb7-482d-bd8d-f2683ab938ad None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:23.651157 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2269f97c-4fb7-482d-bd8d-f2683ab938ad None admin] RBAC: Adding query filter params (name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:23.651565 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2269f97c-4fb7-482d-bd8d-f2683ab938ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:23.655476 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2269f97c-4fb7-482d-bd8d-f2683ab938ad None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:23.670440 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 240/490] 10.222.0.27 () {56 vars in 1103 bytes} [Sun Sep 4 00:48:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:23.676177 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269f97c-4fb7-482d-bd8d-f2683ab938ad None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:23.692300 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df2da71-7278-4043-bb45-77eca30bd181 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:23.733071 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f4c14b-626c-4624-886c-0bfdb011a653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:23.734309 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4c14b-626c-4624-886c-0bfdb011a653 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:23.734740 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4c14b-626c-4624-886c-0bfdb011a653 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:23.735292 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4c14b-626c-4624-886c-0bfdb011a653 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:23.748138 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f4c14b-626c-4624-886c-0bfdb011a653 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:23.749859 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f4c14b-626c-4624-886c-0bfdb011a653 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:23.767799 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-76f4c14b-626c-4624-886c-0bfdb011a653 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:23.767799 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:23.767799 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:23.767799 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:23.767799 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:23.767799 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:23.767799 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:23.767799 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:23.767799 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:23.767799 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:23.767799 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:23.767799 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:23.767799 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:23.767799 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:23.767799 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:23.767799 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:23.769276 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:23.769276 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:23.769276 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:23.769276 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:23.769276 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:23.769276 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:23.769276 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:23.769276 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:23.769276 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:23.769276 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:23.769276 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:23.769276 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:23.769276 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:23.769276 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:23.769276 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:23.769276 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:23.769276 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:23.770095 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:23.770095 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:23.770095 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:23.772278 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 251/491] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:23.777546 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f4c14b-626c-4624-886c-0bfdb011a653 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:23.845161 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:23.848349 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269f97c-4fb7-482d-bd8d-f2683ab938ad None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:23.884630 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2c624-cdfd-475b-b761-9a347139e03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:23.885339 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2c624-cdfd-475b-b761-9a347139e03b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:23.885707 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2c624-cdfd-475b-b761-9a347139e03b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:23.886077 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2c624-cdfd-475b-b761-9a347139e03b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:23.898221 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b2c624-cdfd-475b-b761-9a347139e03b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:23.904392 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b2c624-cdfd-475b-b761-9a347139e03b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:23.915061 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-37b2c624-cdfd-475b-b761-9a347139e03b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:23.915061 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:23.915061 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:23.915061 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:23.915061 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:23.915061 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:23.915061 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:23.915061 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:23.915061 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:23.915061 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:23.915061 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:23.915061 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:23.915061 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:23.915061 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:23.915061 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:23.915061 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:23.916232 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:23.916232 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:23.916232 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:23.916232 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:23.916232 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:23.916232 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:23.916232 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:23.916232 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:23.916232 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:23.916232 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:23.916232 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:23.916232 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:23.916232 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:23.916232 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:23.916232 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:23.916232 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:23.916232 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:23.917101 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:23.917101 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:23.917101 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:23.921346 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 241/492] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:48:23.922700 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc68990-2957-4739-8c0f-99cc2b89d8f3 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:23.923019 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc68990-2957-4739-8c0f-99cc2b89d8f3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:23.923296 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc68990-2957-4739-8c0f-99cc2b89d8f3 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:23.926148 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f4c14b-626c-4624-886c-0bfdb011a653 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:23.927976 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 242/493] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:23.928712 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:23.928850 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc68990-2957-4739-8c0f-99cc2b89d8f3 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:23.970624 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30682b3-2925-4d5f-882e-47ac1b26065d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:23.971267 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30682b3-2925-4d5f-882e-47ac1b26065d None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:23.971553 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30682b3-2925-4d5f-882e-47ac1b26065d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:23.971832 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30682b3-2925-4d5f-882e-47ac1b26065d None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:23.972502 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30682b3-2925-4d5f-882e-47ac1b26065d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:23.972813 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30682b3-2925-4d5f-882e-47ac1b26065d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:23.974656 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30682b3-2925-4d5f-882e-47ac1b26065d None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:23.986646 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 252/494] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:23.988541 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098dddd-8887-4298-89f6-69d159ff2250 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:23.988832 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098dddd-8887-4298-89f6-69d159ff2250 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:23.989123 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098dddd-8887-4298-89f6-69d159ff2250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:24.007356 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-0098dddd-8887-4298-89f6-69d159ff2250 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:24.008601 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-0098dddd-8887-4298-89f6-69d159ff2250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:24.035336 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc68990-2957-4739-8c0f-99cc2b89d8f3 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:24.071599 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d77d1-b0c7-4922-9e1d-eab8a6228c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:24.072192 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d77d1-b0c7-4922-9e1d-eab8a6228c1b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:24.072506 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d77d1-b0c7-4922-9e1d-eab8a6228c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:24.072769 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d77d1-b0c7-4922-9e1d-eab8a6228c1b None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:24.073395 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d77d1-b0c7-4922-9e1d-eab8a6228c1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:24.073724 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d77d1-b0c7-4922-9e1d-eab8a6228c1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:24.074257 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 253/495] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:24.075083 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d77d1-b0c7-4922-9e1d-eab8a6228c1b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:24.075485 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f011d43-cce8-485c-8df2-07a0599dc878 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:24.075769 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f011d43-cce8-485c-8df2-07a0599dc878 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:24.076042 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f011d43-cce8-485c-8df2-07a0599dc878 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:24.077423 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 254/496] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:24.077963 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f011d43-cce8-485c-8df2-07a0599dc878 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:24.098470 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 243/497] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:24.099593 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d77d1-b0c7-4922-9e1d-eab8a6228c1b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:24.115448 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:24.182370 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:24.190814 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f011d43-cce8-485c-8df2-07a0599dc878 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:24.205176 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d77d1-b0c7-4922-9e1d-eab8a6228c1b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:24.234262 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e55c4-0532-41c7-9bd5-61e098da87f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:24.235258 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e55c4-0532-41c7-9bd5-61e098da87f7 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:24.235607 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e55c4-0532-41c7-9bd5-61e098da87f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:24.236030 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e55c4-0532-41c7-9bd5-61e098da87f7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:24.246365 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd547d1-beb1-4b28-9743-4fb262d9ff05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:24.247116 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd547d1-beb1-4b28-9743-4fb262d9ff05 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:24.247338 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e55c4-0532-41c7-9bd5-61e098da87f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:24.247435 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd547d1-beb1-4b28-9743-4fb262d9ff05 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:24.247661 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd547d1-beb1-4b28-9743-4fb262d9ff05 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:24.252220 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-b12e55c4-0532-41c7-9bd5-61e098da87f7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:24.252220 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:24.252220 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:24.252220 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:24.252220 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:24.252220 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:24.252220 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:24.252220 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:24.252220 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:24.252220 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:24.252220 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:24.252220 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:24.252220 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:24.252220 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:24.252220 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:24.252220 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:24.253152 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:24.253152 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:24.253152 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:24.253152 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:24.253152 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:24.253152 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:24.253152 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:24.253152 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:24.253152 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:24.253152 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:24.253152 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:24.253152 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:24.253152 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:24.253152 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:24.253152 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:24.253152 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:24.253912 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:24.253912 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:24.253912 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:24.253912 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:24.253912 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:24.253912 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:24.253912 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:24.253912 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:24.253912 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:24.253912 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:24.253912 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:24.253912 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:24.253912 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 255/498] 10.222.0.27 () {56 vars in 1102 bytes} [Sun Sep 4 00:48:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:24.255383 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57688b4-42e5-4de9-9ebd-69d269ed3403 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:24.255383 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57688b4-42e5-4de9-9ebd-69d269ed3403 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:24.255383 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57688b4-42e5-4de9-9ebd-69d269ed3403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:24.260954 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd547d1-beb1-4b28-9743-4fb262d9ff05 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:24.262409 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd547d1-beb1-4b28-9743-4fb262d9ff05 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:24.273411 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-c57688b4-42e5-4de9-9ebd-69d269ed3403 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:24.274624 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c57688b4-42e5-4de9-9ebd-69d269ed3403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:24.275796 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-cdd547d1-beb1-4b28-9743-4fb262d9ff05 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 04 00:48:24.275796 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:24.275796 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:24.275796 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:24.275796 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:24.275796 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:24.275796 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:24.275796 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:24.275796 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:24.275796 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:24.275796 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:24.275796 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:24.275796 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:48:24.275796 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:48:24.275796 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:48:24.275796 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:48:24.276675 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:24.276675 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:24.276675 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:48:24.276675 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:24.276675 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:48:24.276675 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:24.276675 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:24.276675 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:24.276675 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:24.276675 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:24.276675 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:24.276675 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:24.276675 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:24.276675 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:24.276675 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:24.276675 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:24.276675 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:24.276675 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:24.277618 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:24.277618 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:24.277618 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:48:24.277618 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:48:24.277618 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:48:24.277618 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:48:24.277618 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:48:24.277618 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:48:24.277618 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 04 00:48:24.277618 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:24.277618 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 244/499] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:48:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:24.279144 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd547d1-beb1-4b28-9743-4fb262d9ff05 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:24.351896 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 256/500] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:24.352774 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57688b4-42e5-4de9-9ebd-69d269ed3403 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:24.364717 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:24.395200 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd547d1-beb1-4b28-9743-4fb262d9ff05 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:24.431962 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa0207-2f77-4e4b-8c9c-95e719c4742a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:24.432811 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa0207-2f77-4e4b-8c9c-95e719c4742a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:24.433131 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa0207-2f77-4e4b-8c9c-95e719c4742a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:24.433503 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa0207-2f77-4e4b-8c9c-95e719c4742a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:24.441568 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa0207-2f77-4e4b-8c9c-95e719c4742a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:24.443134 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa0207-2f77-4e4b-8c9c-95e719c4742a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:24.447070 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:24.454348 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-c8aa0207-2f77-4e4b-8c9c-95e719c4742a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 04 00:48:24.454348 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:24.454348 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:24.454348 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:24.454348 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:24.454348 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:24.454348 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:24.454348 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:24.454348 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:24.454348 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:24.454348 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:24.454348 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:24.454348 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:48:24.454348 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:48:24.454348 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:48:24.454348 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:48:24.461249 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:24.461249 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:24.461249 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:48:24.461249 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:24.461249 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:48:24.461249 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:24.461249 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:24.461249 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:24.461249 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:24.461249 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:24.461249 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:24.461249 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:24.461249 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:24.461249 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:24.461249 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:24.461249 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:24.461249 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:24.461249 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:24.462155 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:24.462155 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:24.462155 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:48:24.462155 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:48:24.462155 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:48:24.462155 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:48:24.462155 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:48:24.462155 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:48:24.462155 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 04 00:48:24.462155 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:24.462155 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 245/501] 10.222.0.27 () {56 vars in 1093 bytes} [Sun Sep 4 00:48:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:24.462155 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa0207-2f77-4e4b-8c9c-95e719c4742a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:24.470945 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57688b4-42e5-4de9-9ebd-69d269ed3403 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:24.507490 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30a8f0-331c-4c2d-9b64-05cfcf359427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:24.508313 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30a8f0-331c-4c2d-9b64-05cfcf359427 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:24.508634 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30a8f0-331c-4c2d-9b64-05cfcf359427 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:24.508916 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30a8f0-331c-4c2d-9b64-05cfcf359427 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:24.509587 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c30a8f0-331c-4c2d-9b64-05cfcf359427 None admin] RBAC: Adding query filter params (name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:24.509893 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c30a8f0-331c-4c2d-9b64-05cfcf359427 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:24.511268 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c30a8f0-331c-4c2d-9b64-05cfcf359427 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:24.520134 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 257/502] 10.222.0.27 () {56 vars in 1103 bytes} [Sun Sep 4 00:48:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:24.524870 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30a8f0-331c-4c2d-9b64-05cfcf359427 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:24.581172 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa0207-2f77-4e4b-8c9c-95e719c4742a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:24.613513 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:24.618780 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd048969-bcdd-498f-9a38-7d6f3046956c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:24.619549 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd048969-bcdd-498f-9a38-7d6f3046956c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:24.620027 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd048969-bcdd-498f-9a38-7d6f3046956c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:24.620201 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd048969-bcdd-498f-9a38-7d6f3046956c None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:24.620945 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd048969-bcdd-498f-9a38-7d6f3046956c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:24.621447 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd048969-bcdd-498f-9a38-7d6f3046956c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:24.627021 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd048969-bcdd-498f-9a38-7d6f3046956c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:24.628397 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30a8f0-331c-4c2d-9b64-05cfcf359427 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:24.659757 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 246/503] 10.222.0.27 () {56 vars in 1137 bytes} [Sun Sep 4 00:48:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:24.660658 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd048969-bcdd-498f-9a38-7d6f3046956c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:24.666933 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43aba8-a879-44e9-8d79-111fe2d8a7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:24.667556 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43aba8-a879-44e9-8d79-111fe2d8a7cd None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:24.667832 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43aba8-a879-44e9-8d79-111fe2d8a7cd None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:24.668104 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43aba8-a879-44e9-8d79-111fe2d8a7cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:24.678671 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f43aba8-a879-44e9-8d79-111fe2d8a7cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:24.683891 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-1f43aba8-a879-44e9-8d79-111fe2d8a7cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:24.683891 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:24.683891 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:24.683891 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:24.683891 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:24.683891 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:24.683891 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:24.683891 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:24.683891 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:24.683891 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:24.683891 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:24.683891 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:24.683891 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:24.683891 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:24.683891 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:24.683891 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:24.684956 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:24.684956 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:24.684956 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:24.684956 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:24.684956 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:24.684956 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:24.684956 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:24.684956 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:24.684956 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:24.684956 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:24.684956 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:24.684956 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:24.684956 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:24.684956 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:24.684956 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:24.684956 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:24.685737 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:24.685737 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:24.685737 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:24.685737 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:24.685737 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:24.685737 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:24.685737 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:24.685737 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:24.685737 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:24.685737 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:24.685737 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:24.685737 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:24.685737 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 258/504] 10.222.0.27 () {56 vars in 1102 bytes} [Sun Sep 4 00:48:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:24.685737 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43aba8-a879-44e9-8d79-111fe2d8a7cd None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:24.695719 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:24.767739 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd048969-bcdd-498f-9a38-7d6f3046956c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:24.788159 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43aba8-a879-44e9-8d79-111fe2d8a7cd None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:24.802367 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8c94b-340b-4ea4-81ec-bbaba21268a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:24.802960 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8c94b-340b-4ea4-81ec-bbaba21268a1 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:24.803233 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8c94b-340b-4ea4-81ec-bbaba21268a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:24.803501 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8c94b-340b-4ea4-81ec-bbaba21268a1 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:24.804212 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8c94b-340b-4ea4-81ec-bbaba21268a1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:24.804531 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8c94b-340b-4ea4-81ec-bbaba21268a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:24.805948 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8c94b-340b-4ea4-81ec-bbaba21268a1 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:24.822355 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7ea4c-a9b7-40c6-90b9-9fbc0c25392d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:24.822975 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7ea4c-a9b7-40c6-90b9-9fbc0c25392d None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:24.823177 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 247/505] 10.222.0.27 () {56 vars in 1133 bytes} [Sun Sep 4 00:48:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:24.823266 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7ea4c-a9b7-40c6-90b9-9fbc0c25392d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:24.823524 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7ea4c-a9b7-40c6-90b9-9fbc0c25392d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:24.823981 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8c94b-340b-4ea4-81ec-bbaba21268a1 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:24.838274 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d7ea4c-a9b7-40c6-90b9-9fbc0c25392d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:24.839723 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d7ea4c-a9b7-40c6-90b9-9fbc0c25392d None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:24.854296 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-e1d7ea4c-a9b7-40c6-90b9-9fbc0c25392d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:24.854296 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:24.854296 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:24.854296 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:24.854296 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:24.854296 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:24.854296 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:24.854296 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:24.854296 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:24.854296 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:24.854296 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:24.854296 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:24.854296 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:24.854296 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:24.854296 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:24.854296 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:24.855233 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:24.855233 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:24.855233 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:24.855233 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:24.855233 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:24.855233 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:24.855233 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:24.855233 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:24.855233 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:24.855233 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:24.855233 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:24.855233 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:24.855233 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:24.855233 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:24.855233 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:24.855233 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:24.855233 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:24.856044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:24.856044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:24.856044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:24.856044 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 259/506] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:24.856259 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7ea4c-a9b7-40c6-90b9-9fbc0c25392d None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:24.878739 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:24.915171 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:24.930057 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8c94b-340b-4ea4-81ec-bbaba21268a1 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:24.956978 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7ea4c-a9b7-40c6-90b9-9fbc0c25392d None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:24.963891 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc17f40-e308-4f20-b86f-906eb8f37093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:24.964565 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc17f40-e308-4f20-b86f-906eb8f37093 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:24.964847 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc17f40-e308-4f20-b86f-906eb8f37093 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:24.965122 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc17f40-e308-4f20-b86f-906eb8f37093 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:24.975621 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc17f40-e308-4f20-b86f-906eb8f37093 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:24.977186 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc17f40-e308-4f20-b86f-906eb8f37093 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:24.989826 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e73393-fe05-4fc0-91f3-53ef07386be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:24.990413 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e73393-fe05-4fc0-91f3-53ef07386be3 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:24.990694 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e73393-fe05-4fc0-91f3-53ef07386be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:24.990965 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e73393-fe05-4fc0-91f3-53ef07386be3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:24.991724 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-4bc17f40-e308-4f20-b86f-906eb8f37093 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:24.991724 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:24.991724 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:24.991724 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:24.991724 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:24.991724 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:24.991724 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:24.991724 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:24.991724 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:24.991724 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:24.991724 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:24.991724 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:24.991724 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:24.991724 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:24.991724 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:24.991724 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:24.992625 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:24.992625 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:24.992625 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:24.992625 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:24.992625 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:24.992625 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:24.992625 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:24.992625 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:24.992625 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:24.992625 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:24.992625 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:24.992625 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:24.992625 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:24.992625 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:24.992625 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:24.992625 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:24.992625 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:24.993517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:24.993517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:24.993517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:24.993517 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 248/507] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:24.993776 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc17f40-e308-4f20-b86f-906eb8f37093 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:25.061986 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e73393-fe05-4fc0-91f3-53ef07386be3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:25.063475 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e73393-fe05-4fc0-91f3-53ef07386be3 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:25.090568 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:25.093445 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc17f40-e308-4f20-b86f-906eb8f37093 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:25.121178 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:25.128405 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b47eba7-af6d-40ce-a89c-2d1f6a181299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:25.129058 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47eba7-af6d-40ce-a89c-2d1f6a181299 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:25.129339 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47eba7-af6d-40ce-a89c-2d1f6a181299 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:25.129617 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47eba7-af6d-40ce-a89c-2d1f6a181299 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:25.130274 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b47eba7-af6d-40ce-a89c-2d1f6a181299 None admin] RBAC: Adding query filter params (name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:25.130577 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b47eba7-af6d-40ce-a89c-2d1f6a181299 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:25.131997 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b47eba7-af6d-40ce-a89c-2d1f6a181299 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:25.139820 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 249/508] 10.222.0.27 () {56 vars in 1103 bytes} [Sun Sep 4 00:48:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:25.140645 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b47eba7-af6d-40ce-a89c-2d1f6a181299 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:25.164371 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 260/509] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:48:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:48:25.165260 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e73393-fe05-4fc0-91f3-53ef07386be3 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:25.239898 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b47eba7-af6d-40ce-a89c-2d1f6a181299 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:25.271954 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e73393-fe05-4fc0-91f3-53ef07386be3 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:25.273331 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f3fe97-fb57-42bc-a749-6b78c65a45c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:25.273950 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3fe97-fb57-42bc-a749-6b78c65a45c5 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:25.274234 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3fe97-fb57-42bc-a749-6b78c65a45c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:25.274508 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3fe97-fb57-42bc-a749-6b78c65a45c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:25.305213 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:25.305516 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ab1679-7375-403f-ab2d-5c9e7ab89723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:25.306121 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab1679-7375-403f-ab2d-5c9e7ab89723 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:25.306397 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab1679-7375-403f-ab2d-5c9e7ab89723 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:25.306671 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab1679-7375-403f-ab2d-5c9e7ab89723 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:25.307299 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ab1679-7375-403f-ab2d-5c9e7ab89723 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:25.307594 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ab1679-7375-403f-ab2d-5c9e7ab89723 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:25.309010 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ab1679-7375-403f-ab2d-5c9e7ab89723 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:25.320067 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 261/510] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:25.320917 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ab1679-7375-403f-ab2d-5c9e7ab89723 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:25.341683 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:25.343994 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f3fe97-fb57-42bc-a749-6b78c65a45c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:25.345511 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f3fe97-fb57-42bc-a749-6b78c65a45c5 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:25.423205 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ab1679-7375-403f-ab2d-5c9e7ab89723 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:25.444855 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 250/511] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:48:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:48:25.445791 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f3fe97-fb57-42bc-a749-6b78c65a45c5 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:25.456303 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae566ba-7c3d-42c6-8d02-4f2b2eab4cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:25.456957 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae566ba-7c3d-42c6-8d02-4f2b2eab4cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:25.457237 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae566ba-7c3d-42c6-8d02-4f2b2eab4cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:25.457507 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae566ba-7c3d-42c6-8d02-4f2b2eab4cd0 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:25.458224 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae566ba-7c3d-42c6-8d02-4f2b2eab4cd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:25.458528 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae566ba-7c3d-42c6-8d02-4f2b2eab4cd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:25.459929 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae566ba-7c3d-42c6-8d02-4f2b2eab4cd0 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:25.471919 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 262/512] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:25.472730 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae566ba-7c3d-42c6-8d02-4f2b2eab4cd0 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:25.510008 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:25.551274 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:25.551495 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f3fe97-fb57-42bc-a749-6b78c65a45c5 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:25.572424 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae566ba-7c3d-42c6-8d02-4f2b2eab4cd0 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:25.586298 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce4240f-b14d-4644-a3fa-4552bb713467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:25.586952 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce4240f-b14d-4644-a3fa-4552bb713467 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:25.587228 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce4240f-b14d-4644-a3fa-4552bb713467 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:25.587497 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce4240f-b14d-4644-a3fa-4552bb713467 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:25.598977 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce4240f-b14d-4644-a3fa-4552bb713467 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:25.600430 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce4240f-b14d-4644-a3fa-4552bb713467 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:25.607306 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65304b62-b4aa-4c92-82b2-2486fde33e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:25.607924 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65304b62-b4aa-4c92-82b2-2486fde33e9e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:25.608203 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65304b62-b4aa-4c92-82b2-2486fde33e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:25.608496 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65304b62-b4aa-4c92-82b2-2486fde33e9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:25.614848 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-9ce4240f-b14d-4644-a3fa-4552bb713467 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:25.614848 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:25.614848 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:25.614848 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:25.614848 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:25.614848 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:25.614848 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:25.614848 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:25.614848 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:25.614848 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:25.614848 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:25.614848 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:25.614848 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:25.614848 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:25.614848 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:25.614848 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:25.616235 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:25.616235 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:25.616235 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:25.616235 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:25.616235 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:25.616235 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:25.616235 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:25.616235 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:25.616235 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:25.616235 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:25.616235 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:25.616235 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:25.616235 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:25.616235 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:25.616235 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:25.616235 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:25.616235 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:25.617321 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:25.617321 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:25.617321 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:25.617321 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 251/513] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:25.618106 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce4240f-b14d-4644-a3fa-4552bb713467 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:25.622451 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65304b62-b4aa-4c92-82b2-2486fde33e9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:25.623933 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65304b62-b4aa-4c92-82b2-2486fde33e9e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:25.640314 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-65304b62-b4aa-4c92-82b2-2486fde33e9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:25.640314 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:25.640314 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:25.640314 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:25.640314 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:25.640314 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:25.640314 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:25.640314 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:25.640314 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:25.640314 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:25.640314 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:25.640314 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:25.640314 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:25.640314 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:25.640314 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:25.640314 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:25.641677 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:25.641677 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:25.641677 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:25.641677 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:25.641677 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:25.641677 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:25.641677 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:25.641677 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:25.641677 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:25.641677 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:25.641677 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:25.641677 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:25.641677 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:25.641677 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:25.641677 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:25.641677 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:25.641677 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:25.642511 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:25.642511 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:25.642511 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:25.642511 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 263/514] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:25.642511 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65304b62-b4aa-4c92-82b2-2486fde33e9e None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:25.719443 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce4240f-b14d-4644-a3fa-4552bb713467 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:25.732883 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:25.741902 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65304b62-b4aa-4c92-82b2-2486fde33e9e None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:25.752848 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75459c8c-8cbf-4638-9e0d-64ad7d0504ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:25.753567 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75459c8c-8cbf-4638-9e0d-64ad7d0504ef None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:25.754093 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75459c8c-8cbf-4638-9e0d-64ad7d0504ef None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:25.754580 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75459c8c-8cbf-4638-9e0d-64ad7d0504ef None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:25.762123 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75459c8c-8cbf-4638-9e0d-64ad7d0504ef None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:25.763503 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75459c8c-8cbf-4638-9e0d-64ad7d0504ef None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:25.770178 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:25.775082 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-75459c8c-8cbf-4638-9e0d-64ad7d0504ef None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 04 00:48:25.775082 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:25.775082 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:25.775082 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:25.775082 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:25.775082 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:25.775082 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:25.775082 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:25.775082 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:25.775082 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:25.775082 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:25.775082 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:25.775082 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:48:25.775082 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:48:25.775082 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:48:25.775082 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:48:25.776147 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:25.776147 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:25.776147 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:48:25.776147 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:25.776147 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:48:25.776147 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:25.776147 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:25.776147 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:25.776147 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:25.776147 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:25.776147 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:25.776147 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:25.776147 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:25.776147 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:25.776147 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:25.776147 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:25.776147 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:25.776147 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:25.777351 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:25.777351 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:25.777351 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:48:25.777351 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:48:25.777351 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:48:25.777351 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:48:25.777351 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:48:25.777351 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:48:25.777351 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 04 00:48:25.777351 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:25.777351 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 252/515] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:48:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:25.777351 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75459c8c-8cbf-4638-9e0d-64ad7d0504ef None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:25.778805 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c3997b-5d87-4d7c-a591-872a6409b806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:25.779432 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c3997b-5d87-4d7c-a591-872a6409b806 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:25.779712 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c3997b-5d87-4d7c-a591-872a6409b806 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:25.779983 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c3997b-5d87-4d7c-a591-872a6409b806 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:25.791291 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c3997b-5d87-4d7c-a591-872a6409b806 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:25.792773 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c3997b-5d87-4d7c-a591-872a6409b806 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:25.806486 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-d1c3997b-5d87-4d7c-a591-872a6409b806 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:25.806486 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:25.806486 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:25.806486 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:25.806486 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:25.806486 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:25.806486 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:25.806486 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:25.806486 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:25.806486 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:25.806486 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:25.806486 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:25.806486 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:25.806486 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:25.806486 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:25.806486 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:25.807354 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:25.807354 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:25.807354 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:25.807354 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:25.807354 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:25.807354 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:25.807354 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:25.807354 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:25.807354 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:25.807354 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:25.807354 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:25.807354 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:25.807354 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:25.807354 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:25.807354 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:25.807354 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:25.807354 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:25.808192 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:25.808192 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:25.808192 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:25.811968 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 264/516] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:25.812803 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c3997b-5d87-4d7c-a591-872a6409b806 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:25.877038 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75459c8c-8cbf-4638-9e0d-64ad7d0504ef None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:25.911467 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c2b8f-ec19-442f-a28f-00d44da0e964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:25.912058 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c2b8f-ec19-442f-a28f-00d44da0e964 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:25.912339 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c2b8f-ec19-442f-a28f-00d44da0e964 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:25.912646 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c2b8f-ec19-442f-a28f-00d44da0e964 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:25.912977 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c3997b-5d87-4d7c-a591-872a6409b806 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:25.913286 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c2b8f-ec19-442f-a28f-00d44da0e964 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:25.914712 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c2b8f-ec19-442f-a28f-00d44da0e964 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:25.946478 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090abc5-a208-4513-bd13-b84cea57f9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:25.947077 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090abc5-a208-4513-bd13-b84cea57f9ee None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:25.947367 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090abc5-a208-4513-bd13-b84cea57f9ee None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:25.947644 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090abc5-a208-4513-bd13-b84cea57f9ee None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:25.948338 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3090abc5-a208-4513-bd13-b84cea57f9ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:25.948676 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3090abc5-a208-4513-bd13-b84cea57f9ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:25.950041 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3090abc5-a208-4513-bd13-b84cea57f9ee None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:25.962040 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 265/517] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:25.962822 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090abc5-a208-4513-bd13-b84cea57f9ee None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:26.005019 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:26.016312 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 253/518] 10.222.0.27 () {60 vars in 1132 bytes} [Sun Sep 4 00:48:25 2022] POST /identity/v3/users => generated 247 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:48:26.017010 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c2b8f-ec19-442f-a28f-00d44da0e964 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:26.053563 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:26.068193 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090abc5-a208-4513-bd13-b84cea57f9ee None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:26.110493 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb776f7c-a5e8-46e2-929c-bc9f01b6d643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:26.111110 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb776f7c-a5e8-46e2-929c-bc9f01b6d643 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:26.111386 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb776f7c-a5e8-46e2-929c-bc9f01b6d643 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:26.112202 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb776f7c-a5e8-46e2-929c-bc9f01b6d643 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:26.112914 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb776f7c-a5e8-46e2-929c-bc9f01b6d643 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:26.113203 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb776f7c-a5e8-46e2-929c-bc9f01b6d643 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:26.114714 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb776f7c-a5e8-46e2-929c-bc9f01b6d643 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:26.131959 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 266/519] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:26.131959 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb776f7c-a5e8-46e2-929c-bc9f01b6d643 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:26.138592 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c2b8f-ec19-442f-a28f-00d44da0e964 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:26.180098 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da15a0-3f72-4763-a190-de4c01c10e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:26.180747 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da15a0-3f72-4763-a190-de4c01c10e5d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:26.181022 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da15a0-3f72-4763-a190-de4c01c10e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:26.181297 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da15a0-3f72-4763-a190-de4c01c10e5d None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:26.182171 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da15a0-3f72-4763-a190-de4c01c10e5d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:26.182341 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da15a0-3f72-4763-a190-de4c01c10e5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:26.183718 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da15a0-3f72-4763-a190-de4c01c10e5d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:26.204680 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 254/520] 10.222.0.27 () {56 vars in 1137 bytes} [Sun Sep 4 00:48:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:26.205446 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da15a0-3f72-4763-a190-de4c01c10e5d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:26.250805 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb776f7c-a5e8-46e2-929c-bc9f01b6d643 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:26.253487 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:26.289611 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac484d-b165-438d-855b-8560ecdfe0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:26.290147 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac484d-b165-438d-855b-8560ecdfe0b6 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:26.290683 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac484d-b165-438d-855b-8560ecdfe0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:26.290959 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac484d-b165-438d-855b-8560ecdfe0b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:26.291599 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ac484d-b165-438d-855b-8560ecdfe0b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:26.291904 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ac484d-b165-438d-855b-8560ecdfe0b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:26.293449 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ac484d-b165-438d-855b-8560ecdfe0b6 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:26.300707 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:26.306940 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 267/521] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:26.308004 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac484d-b165-438d-855b-8560ecdfe0b6 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:26.319239 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da15a0-3f72-4763-a190-de4c01c10e5d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:26.357186 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f73b0f-1438-467d-9b88-2e095e409518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:26.357879 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f73b0f-1438-467d-9b88-2e095e409518 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:26.358156 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f73b0f-1438-467d-9b88-2e095e409518 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:26.358433 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f73b0f-1438-467d-9b88-2e095e409518 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:26.366990 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f73b0f-1438-467d-9b88-2e095e409518 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:26.368292 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f73b0f-1438-467d-9b88-2e095e409518 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:26.380194 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-c3f73b0f-1438-467d-9b88-2e095e409518 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 04 00:48:26.380194 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:26.380194 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:26.380194 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:26.380194 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:26.380194 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:26.380194 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:26.380194 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:26.380194 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:26.380194 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:26.380194 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:26.380194 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:26.380194 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:48:26.380194 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:48:26.380194 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:48:26.380194 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:48:26.381212 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:26.381212 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:26.381212 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:48:26.381212 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:26.381212 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:48:26.381212 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:26.381212 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:26.381212 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:26.381212 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:26.381212 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:26.381212 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:26.381212 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:26.381212 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:26.381212 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:26.381212 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:26.381212 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:26.381212 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:26.381212 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:26.382233 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:26.382233 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:26.382233 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:48:26.382233 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:48:26.382233 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:48:26.382233 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:48:26.382233 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:48:26.382233 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:48:26.382233 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 04 00:48:26.382233 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:26.382233 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 255/522] 10.222.0.27 () {56 vars in 1102 bytes} [Sun Sep 4 00:48:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:26.382233 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f73b0f-1438-467d-9b88-2e095e409518 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:26.423339 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac484d-b165-438d-855b-8560ecdfe0b6 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:26.458504 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ddc7a-b710-433e-ab1b-ef2e0b9831e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:26.459138 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ddc7a-b710-433e-ab1b-ef2e0b9831e5 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:26.459424 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ddc7a-b710-433e-ab1b-ef2e0b9831e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:26.459687 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ddc7a-b710-433e-ab1b-ef2e0b9831e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:26.466549 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497ddc7a-b710-433e-ab1b-ef2e0b9831e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:26.468013 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497ddc7a-b710-433e-ab1b-ef2e0b9831e5 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:26.481379 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-497ddc7a-b710-433e-ab1b-ef2e0b9831e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:26.481379 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:26.481379 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:26.481379 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:26.481379 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:26.481379 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:26.481379 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:26.481379 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:26.481379 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:26.481379 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:26.481379 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:26.481379 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:26.481379 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:48:26.481379 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:48:26.481379 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:48:26.482253 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:48:26.482253 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:26.482253 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:26.482253 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:26.482253 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:26.482253 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:26.482253 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:26.482253 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:26.482253 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:26.482253 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:26.482253 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:26.482253 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:26.482253 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:26.482253 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:26.482253 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:26.482253 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:26.482253 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:26.483069 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:48:26.483069 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:48:26.483069 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:48:26.483069 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:48:26.483069 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:48:26.483069 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:48:26.483069 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:26.483069 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:26.483069 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 268/523] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:48:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:26.483632 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ddc7a-b710-433e-ab1b-ef2e0b9831e5 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:26.501005 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f73b0f-1438-467d-9b88-2e095e409518 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:26.511650 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:26.539468 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6bf9bf-207c-450b-a89c-ed84b3474698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:26.540128 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6bf9bf-207c-450b-a89c-ed84b3474698 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:26.540700 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6bf9bf-207c-450b-a89c-ed84b3474698 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:26.540987 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6bf9bf-207c-450b-a89c-ed84b3474698 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:26.559629 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:26.597616 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ddc7a-b710-433e-ab1b-ef2e0b9831e5 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:26.614401 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6bf9bf-207c-450b-a89c-ed84b3474698 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:26.615989 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6bf9bf-207c-450b-a89c-ed84b3474698 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:26.634098 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c7bdd-6ef4-4647-851d-494001b539c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:26.634721 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c7bdd-6ef4-4647-851d-494001b539c1 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:26.635001 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c7bdd-6ef4-4647-851d-494001b539c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:26.635275 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c7bdd-6ef4-4647-851d-494001b539c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:26.641799 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748c7bdd-6ef4-4647-851d-494001b539c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:26.643828 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748c7bdd-6ef4-4647-851d-494001b539c1 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:26.655457 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-748c7bdd-6ef4-4647-851d-494001b539c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:26.655457 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:26.655457 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:26.655457 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:26.655457 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:26.655457 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:26.655457 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:26.655457 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:26.655457 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:26.655457 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:26.655457 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:26.655457 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:26.655457 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:48:26.655457 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:48:26.655457 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:48:26.656319 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:48:26.656319 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:26.656319 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:26.656319 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:26.656319 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:26.656319 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:26.656319 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:26.656319 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:26.656319 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:26.656319 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:26.656319 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:26.656319 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:26.656319 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:26.656319 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:26.656319 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:26.656319 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:26.656319 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:26.657472 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:48:26.657472 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:48:26.657472 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:48:26.657472 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:48:26.657472 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:48:26.657472 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:48:26.657472 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:26.657472 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:26.661345 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 269/524] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:48:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:26.662056 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c7bdd-6ef4-4647-851d-494001b539c1 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:26.723923 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 256/525] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:48:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:48:26.725220 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6bf9bf-207c-450b-a89c-ed84b3474698 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:26.754665 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:26.777879 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c7bdd-6ef4-4647-851d-494001b539c1 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:26.779767 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:26.823469 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1325b6-d27d-42fa-a7ce-804023543657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:26.824038 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1325b6-d27d-42fa-a7ce-804023543657 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:26.824320 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1325b6-d27d-42fa-a7ce-804023543657 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:26.824620 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1325b6-d27d-42fa-a7ce-804023543657 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:26.825353 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1325b6-d27d-42fa-a7ce-804023543657 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:26.825661 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1325b6-d27d-42fa-a7ce-804023543657 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:26.827097 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1325b6-d27d-42fa-a7ce-804023543657 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:26.836780 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6bf9bf-207c-450b-a89c-ed84b3474698 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:26.868130 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 270/526] 10.222.0.27 () {56 vars in 1139 bytes} [Sun Sep 4 00:48:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:26.868130 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1325b6-d27d-42fa-a7ce-804023543657 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:26.880294 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276989c9-103e-447f-8b01-94a6b56b39fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:26.880948 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276989c9-103e-447f-8b01-94a6b56b39fa None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:26.881235 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276989c9-103e-447f-8b01-94a6b56b39fa None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:26.881508 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276989c9-103e-447f-8b01-94a6b56b39fa None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:26.889702 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276989c9-103e-447f-8b01-94a6b56b39fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:26.889702 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276989c9-103e-447f-8b01-94a6b56b39fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:26.889702 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276989c9-103e-447f-8b01-94a6b56b39fa None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:26.908120 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 257/527] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:48:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:26.908120 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276989c9-103e-447f-8b01-94a6b56b39fa None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:26.986420 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1325b6-d27d-42fa-a7ce-804023543657 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:27.014281 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276989c9-103e-447f-8b01-94a6b56b39fa None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:27.024868 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:27.028448 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c3608-17ec-4ae3-ae5d-80eb6a9bae4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:27.029094 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c3608-17ec-4ae3-ae5d-80eb6a9bae4d None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:27.029372 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c3608-17ec-4ae3-ae5d-80eb6a9bae4d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:27.029640 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c3608-17ec-4ae3-ae5d-80eb6a9bae4d None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:27.030337 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85c3608-17ec-4ae3-ae5d-80eb6a9bae4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:27.030636 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85c3608-17ec-4ae3-ae5d-80eb6a9bae4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:27.032078 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85c3608-17ec-4ae3-ae5d-80eb6a9bae4d None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:27.047435 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:27.056363 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 271/528] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:48:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:27.056554 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f56ec-7e66-453f-a121-171ed1bb9f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:27.056554 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f56ec-7e66-453f-a121-171ed1bb9f79 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:27.056554 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f56ec-7e66-453f-a121-171ed1bb9f79 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:27.056821 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c3608-17ec-4ae3-ae5d-80eb6a9bae4d None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:27.056914 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f56ec-7e66-453f-a121-171ed1bb9f79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:27.069734 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652f56ec-7e66-453f-a121-171ed1bb9f79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:27.071523 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652f56ec-7e66-453f-a121-171ed1bb9f79 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:27.087730 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-652f56ec-7e66-453f-a121-171ed1bb9f79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:27.087730 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:27.087730 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:27.087730 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:27.087730 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:27.087730 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:27.087730 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:27.087730 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:27.087730 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:27.087730 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:27.087730 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:27.087730 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:27.087730 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:27.087730 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:27.087730 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:27.087730 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:27.088691 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:27.088691 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:27.088691 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:27.088691 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:27.088691 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:27.088691 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:27.088691 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:27.088691 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:27.088691 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:27.088691 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:27.088691 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:27.088691 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:27.088691 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:27.088691 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:27.088691 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:27.088691 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:27.088691 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:27.089503 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:27.089503 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:27.089503 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:27.089503 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 258/529] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:27.096688 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f56ec-7e66-453f-a121-171ed1bb9f79 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:27.180136 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c3608-17ec-4ae3-ae5d-80eb6a9bae4d None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:27.222665 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f56ec-7e66-453f-a121-171ed1bb9f79 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:27.222883 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36255e2e-b035-434e-92bd-962b91740267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:27.223332 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36255e2e-b035-434e-92bd-962b91740267 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:27.223616 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36255e2e-b035-434e-92bd-962b91740267 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:27.223889 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36255e2e-b035-434e-92bd-962b91740267 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:27.262119 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e254e6-1ecf-483a-9b14-8d7902e876f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:27.262716 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e254e6-1ecf-483a-9b14-8d7902e876f8 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:27.263287 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e254e6-1ecf-483a-9b14-8d7902e876f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:27.263566 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e254e6-1ecf-483a-9b14-8d7902e876f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:27.264289 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e254e6-1ecf-483a-9b14-8d7902e876f8 None admin] RBAC: Adding query filter params (role.id=6e3e398ba29e436e8532a45d2fdf1321, scope.project.id=5100a8d316d243fb8f82e9044e1b3b06, user.id=1179503bdab547c99175a27b7ce27b9d) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:27.264752 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e254e6-1ecf-483a-9b14-8d7902e876f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:27.266654 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e254e6-1ecf-483a-9b14-8d7902e876f8 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:27.280511 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 259/530] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:48:27 2022] GET /identity/v3/role_assignments?user.id=1179503bdab547c99175a27b7ce27b9d&scope.project.id=5100a8d316d243fb8f82e9044e1b3b06&role.id=6e3e398ba29e436e8532a45d2fdf1321 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:27.280511 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e254e6-1ecf-483a-9b14-8d7902e876f8 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:27.314161 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:27.323058 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:27.326052 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36255e2e-b035-434e-92bd-962b91740267 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:27.327935 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36255e2e-b035-434e-92bd-962b91740267 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:27.436384 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e254e6-1ecf-483a-9b14-8d7902e876f8 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:27.472777 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 272/531] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:48:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:48:27.474293 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36255e2e-b035-434e-92bd-962b91740267 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:27.493235 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b55c5-7791-4bab-be12-44bd1f479e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:27.493529 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b55c5-7791-4bab-be12-44bd1f479e50 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:27.496430 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b55c5-7791-4bab-be12-44bd1f479e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:27.496430 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b55c5-7791-4bab-be12-44bd1f479e50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:27.496430 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230b55c5-7791-4bab-be12-44bd1f479e50 None admin] RBAC: Adding query filter params (role.id=6e3e398ba29e436e8532a45d2fdf1321, scope.project.id=5100a8d316d243fb8f82e9044e1b3b06, user.id=68bc00bda5194cd8a7928f8c4df4ccd1) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:27.496430 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230b55c5-7791-4bab-be12-44bd1f479e50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:27.496724 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230b55c5-7791-4bab-be12-44bd1f479e50 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:27.504408 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 260/532] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:48:27 2022] GET /identity/v3/role_assignments?user.id=68bc00bda5194cd8a7928f8c4df4ccd1&scope.project.id=5100a8d316d243fb8f82e9044e1b3b06&role.id=6e3e398ba29e436e8532a45d2fdf1321 => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:27.505304 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b55c5-7791-4bab-be12-44bd1f479e50 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:27.665016 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:27.670159 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36255e2e-b035-434e-92bd-962b91740267 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:27.704624 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:27.732793 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f39063-738b-4028-b437-bb3025a03ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:27.734501 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f39063-738b-4028-b437-bb3025a03ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:27.736570 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b55c5-7791-4bab-be12-44bd1f479e50 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:27.737262 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f39063-738b-4028-b437-bb3025a03ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:27.737902 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f39063-738b-4028-b437-bb3025a03ae6 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:27.738764 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f39063-738b-4028-b437-bb3025a03ae6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:27.739836 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f39063-738b-4028-b437-bb3025a03ae6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:27.743712 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f39063-738b-4028-b437-bb3025a03ae6 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:27.768717 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 273/533] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:27.776528 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f39063-738b-4028-b437-bb3025a03ae6 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:27.813135 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509f7a0-3c8c-416b-b995-7d0c4379910c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:27.813777 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509f7a0-3c8c-416b-b995-7d0c4379910c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:27.814062 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509f7a0-3c8c-416b-b995-7d0c4379910c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:27.814335 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509f7a0-3c8c-416b-b995-7d0c4379910c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:27.834901 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509f7a0-3c8c-416b-b995-7d0c4379910c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:27.836383 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509f7a0-3c8c-416b-b995-7d0c4379910c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:27.860491 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-9509f7a0-3c8c-416b-b995-7d0c4379910c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:27.860491 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:27.860491 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:27.860491 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:27.860491 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:27.860491 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:27.860491 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:27.860491 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:27.860491 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:27.860491 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:27.860491 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:27.860491 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:27.860491 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:27.860491 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:27.860491 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:27.860491 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:27.861452 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:27.861452 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:27.861452 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:27.861452 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:27.861452 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:27.861452 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:27.861452 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:27.861452 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:27.861452 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:27.861452 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:27.861452 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:27.861452 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:27.861452 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:27.861452 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:27.861452 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:27.861452 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:27.861452 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:27.863262 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:27.863262 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:27.863262 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:27.863262 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 261/534] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:27.868424 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509f7a0-3c8c-416b-b995-7d0c4379910c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:27.986411 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509f7a0-3c8c-416b-b995-7d0c4379910c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:28.025079 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947bcaab-d816-4de9-9e48-7dc78a98469c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:28.025861 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947bcaab-d816-4de9-9e48-7dc78a98469c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:28.026157 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947bcaab-d816-4de9-9e48-7dc78a98469c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:28.026438 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947bcaab-d816-4de9-9e48-7dc78a98469c None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:28.027158 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947bcaab-d816-4de9-9e48-7dc78a98469c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:28.027468 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947bcaab-d816-4de9-9e48-7dc78a98469c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:28.029179 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947bcaab-d816-4de9-9e48-7dc78a98469c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:28.043703 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 262/535] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:48:28.049233 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947bcaab-d816-4de9-9e48-7dc78a98469c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:28.052811 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:28.054456 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f39063-738b-4028-b437-bb3025a03ae6 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:28.133998 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e33f2b-aa57-4610-8e25-74ee4f231208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:28.134689 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e33f2b-aa57-4610-8e25-74ee4f231208 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:28.134987 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e33f2b-aa57-4610-8e25-74ee4f231208 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:28.135483 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e33f2b-aa57-4610-8e25-74ee4f231208 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:28.149512 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e33f2b-aa57-4610-8e25-74ee4f231208 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:28.150983 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e33f2b-aa57-4610-8e25-74ee4f231208 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:28.170467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-81e33f2b-aa57-4610-8e25-74ee4f231208 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:28.170467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:28.170467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:28.170467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:28.170467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:28.170467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:28.170467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:28.170467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:28.170467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:28.170467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:28.170467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:28.170467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:28.170467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:48:28.170467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:48:28.170467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:48:28.171509 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:48:28.171509 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:28.171509 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:28.171509 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:28.171509 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:28.171509 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:28.171509 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:28.171509 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:28.171509 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:28.171509 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:28.171509 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:28.171509 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:28.171509 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:28.171509 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:28.171509 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:28.171509 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:28.171509 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:28.172357 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:48:28.172357 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:48:28.172357 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:48:28.172357 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:48:28.172357 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:48:28.172357 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:48:28.172357 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:28.172357 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:28.172357 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 274/536] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:48:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 399 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:28.177364 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e33f2b-aa57-4610-8e25-74ee4f231208 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:28.210671 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:28.235718 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947bcaab-d816-4de9-9e48-7dc78a98469c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:28.297033 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7367e08b-e4ad-4f49-8081-9a1426cc4f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:28.298566 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367e08b-e4ad-4f49-8081-9a1426cc4f29 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:28.298851 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367e08b-e4ad-4f49-8081-9a1426cc4f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:28.299651 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367e08b-e4ad-4f49-8081-9a1426cc4f29 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:28.308787 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7367e08b-e4ad-4f49-8081-9a1426cc4f29 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:28.310660 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7367e08b-e4ad-4f49-8081-9a1426cc4f29 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:28.326430 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-7367e08b-e4ad-4f49-8081-9a1426cc4f29 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:28.326430 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:28.326430 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:28.326430 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:28.326430 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:28.326430 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:28.326430 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:28.326430 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:28.326430 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:28.326430 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:28.326430 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:28.326430 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:28.326430 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:48:28.326430 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:48:28.326430 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:48:28.327507 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:48:28.327507 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:28.327507 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:28.327507 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:28.327507 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:28.327507 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:28.327507 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:28.327507 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:28.327507 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:28.327507 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:28.327507 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:28.327507 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:28.327507 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:28.327507 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:28.327507 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:28.327507 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:28.327507 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:28.328601 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:48:28.328601 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:48:28.328601 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:48:28.328601 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:48:28.328601 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:48:28.328601 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:48:28.328601 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:28.328601 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:28.328601 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 263/537] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:48:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:48:28.329602 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86712774-652d-4014-a977-152f4ffb974d None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:28.329602 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86712774-652d-4014-a977-152f4ffb974d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:28.332684 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86712774-652d-4014-a977-152f4ffb974d None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:28.334631 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 264/538] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:28.335413 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86712774-652d-4014-a977-152f4ffb974d None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:28.347096 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e33f2b-aa57-4610-8e25-74ee4f231208 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:28.395406 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d982090-ff98-4c2e-97a9-04ab7b34493d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:28.396027 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d982090-ff98-4c2e-97a9-04ab7b34493d None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:28.396311 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d982090-ff98-4c2e-97a9-04ab7b34493d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:28.396678 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d982090-ff98-4c2e-97a9-04ab7b34493d None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:28.397382 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d982090-ff98-4c2e-97a9-04ab7b34493d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:28.398234 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d982090-ff98-4c2e-97a9-04ab7b34493d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:28.399983 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d982090-ff98-4c2e-97a9-04ab7b34493d None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:28.404827 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:28.417509 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 275/539] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:48:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:28.418953 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90ab07-c041-4dea-951b-f64d7eb7f66d None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:28.419239 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90ab07-c041-4dea-951b-f64d7eb7f66d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:28.419563 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90ab07-c041-4dea-951b-f64d7eb7f66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:28.438741 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-2f90ab07-c041-4dea-951b-f64d7eb7f66d None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:28.440652 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2f90ab07-c041-4dea-951b-f64d7eb7f66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:28.468599 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86712774-652d-4014-a977-152f4ffb974d None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:28.519740 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaaa0a1-b49f-429e-b895-f1f2402370fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:28.520467 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaaa0a1-b49f-429e-b895-f1f2402370fc None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:28.520852 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaaa0a1-b49f-429e-b895-f1f2402370fc None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:28.521190 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaaa0a1-b49f-429e-b895-f1f2402370fc None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:28.521944 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaaa0a1-b49f-429e-b895-f1f2402370fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:28.522385 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaaa0a1-b49f-429e-b895-f1f2402370fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:28.523964 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaaa0a1-b49f-429e-b895-f1f2402370fc None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:28.528624 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 276/540] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:28.530532 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90ab07-c041-4dea-951b-f64d7eb7f66d None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:28.550434 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 265/541] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:48:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:28.556259 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:28.556966 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaaa0a1-b49f-429e-b895-f1f2402370fc None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:28.720123 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90ab07-c041-4dea-951b-f64d7eb7f66d None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:28.732682 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaaa0a1-b49f-429e-b895-f1f2402370fc None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:28.767171 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:28.792497 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf8052-c55d-4c51-9a51-97aef8dde91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:28.794357 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf8052-c55d-4c51-9a51-97aef8dde91d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:28.795919 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf8052-c55d-4c51-9a51-97aef8dde91d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:28.797119 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf8052-c55d-4c51-9a51-97aef8dde91d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:28.798121 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf8052-c55d-4c51-9a51-97aef8dde91d None admin] RBAC: Adding query filter params (role.id=6e3e398ba29e436e8532a45d2fdf1321, scope.project.id=5100a8d316d243fb8f82e9044e1b3b06, user.id=861c278f8bf34b3bb407b2e051119ee9) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:28.800144 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf8052-c55d-4c51-9a51-97aef8dde91d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:28.801896 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf8052-c55d-4c51-9a51-97aef8dde91d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:28.807710 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d836c-8746-45ee-9f7f-e6169e9338b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:28.808628 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d836c-8746-45ee-9f7f-e6169e9338b4 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:28.809112 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d836c-8746-45ee-9f7f-e6169e9338b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:28.809591 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d836c-8746-45ee-9f7f-e6169e9338b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:28.810520 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742d836c-8746-45ee-9f7f-e6169e9338b4 None admin] RBAC: Adding query filter params (role.id=6e3e398ba29e436e8532a45d2fdf1321, scope.project.id=5100a8d316d243fb8f82e9044e1b3b06, user.id=6297dbc521e94330ae9294b942144181) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:28.811030 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742d836c-8746-45ee-9f7f-e6169e9338b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:28.812986 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742d836c-8746-45ee-9f7f-e6169e9338b4 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:28.818145 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 266/542] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:48:28 2022] GET /identity/v3/role_assignments?user.id=861c278f8bf34b3bb407b2e051119ee9&scope.project.id=5100a8d316d243fb8f82e9044e1b3b06&role.id=6e3e398ba29e436e8532a45d2fdf1321 => generated 259 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:28.820258 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 277/543] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:48:28 2022] GET /identity/v3/role_assignments?user.id=6297dbc521e94330ae9294b942144181&scope.project.id=5100a8d316d243fb8f82e9044e1b3b06&role.id=6e3e398ba29e436e8532a45d2fdf1321 => generated 259 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:28.934867 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf8052-c55d-4c51-9a51-97aef8dde91d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:29.105358 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf8052-c55d-4c51-9a51-97aef8dde91d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:29.156191 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f67934-c215-4e7e-b309-404d2ad6efd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:29.156848 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f67934-c215-4e7e-b309-404d2ad6efd3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:29.157128 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f67934-c215-4e7e-b309-404d2ad6efd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:29.157401 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f67934-c215-4e7e-b309-404d2ad6efd3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:29.171320 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f67934-c215-4e7e-b309-404d2ad6efd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:29.177249 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-a8f67934-c215-4e7e-b309-404d2ad6efd3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:29.177249 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:29.177249 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:29.177249 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:29.177249 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:29.177249 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:29.177249 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:29.177249 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:29.177249 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:29.177249 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:29.177249 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:29.177249 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:29.177249 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:29.177249 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:29.177249 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:29.177249 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:29.178125 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:29.178125 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:29.178125 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:29.178125 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:29.178125 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:29.178125 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:29.178125 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:29.178125 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:29.178125 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:29.178125 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:29.178125 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:29.178125 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:29.178125 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:29.178125 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:29.178125 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:29.178125 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:29.178921 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:29.178921 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:29.178921 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:29.178921 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:29.178921 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:29.178921 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:29.178921 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:29.178921 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:29.178921 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:29.178921 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:29.178921 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:29.178921 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:29.178921 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 267/544] 10.222.0.27 () {56 vars in 1102 bytes} [Sun Sep 4 00:48:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:29.184160 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d836c-8746-45ee-9f7f-e6169e9338b4 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:29.319224 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:29.355946 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d836c-8746-45ee-9f7f-e6169e9338b4 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:29.407081 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b84fa-7120-4fc5-bbca-983bf82fae7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:29.407702 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b84fa-7120-4fc5-bbca-983bf82fae7a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:29.407985 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b84fa-7120-4fc5-bbca-983bf82fae7a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:29.408261 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b84fa-7120-4fc5-bbca-983bf82fae7a None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:29.409507 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161b84fa-7120-4fc5-bbca-983bf82fae7a None admin] RBAC: Adding query filter params (name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:29.409813 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161b84fa-7120-4fc5-bbca-983bf82fae7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:29.411269 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161b84fa-7120-4fc5-bbca-983bf82fae7a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:29.420709 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 278/545] 10.222.0.27 () {56 vars in 1103 bytes} [Sun Sep 4 00:48:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:29.426581 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f67934-c215-4e7e-b309-404d2ad6efd3 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:29.443217 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c4c7f-6f39-4c8a-a4d4-e4594de4863e None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:29.443504 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c4c7f-6f39-4c8a-a4d4-e4594de4863e None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:29.443775 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c4c7f-6f39-4c8a-a4d4-e4594de4863e None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:29.445378 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 279/546] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 00:48:29.450056 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52709ed-5e92-49cd-81b5-1fe5a1801f80 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:29.450338 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52709ed-5e92-49cd-81b5-1fe5a1801f80 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:29.450610 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52709ed-5e92-49cd-81b5-1fe5a1801f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:29.470207 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-b52709ed-5e92-49cd-81b5-1fe5a1801f80 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:29.471894 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b52709ed-5e92-49cd-81b5-1fe5a1801f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:29.509580 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:29.546123 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 280/547] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 00:48:29.616065 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f67934-c215-4e7e-b309-404d2ad6efd3 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:29.650771 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d103f-69e4-464b-8079-8a238a055b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:29.651401 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d103f-69e4-464b-8079-8a238a055b61 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:29.651681 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d103f-69e4-464b-8079-8a238a055b61 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:29.651955 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d103f-69e4-464b-8079-8a238a055b61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:29.662725 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180d103f-69e4-464b-8079-8a238a055b61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:29.664173 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180d103f-69e4-464b-8079-8a238a055b61 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:29.678628 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-180d103f-69e4-464b-8079-8a238a055b61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:29.678628 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:29.678628 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:29.678628 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:29.678628 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:29.678628 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:29.678628 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:29.678628 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:29.678628 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:29.678628 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:29.678628 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:29.678628 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:29.678628 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:29.678628 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:29.678628 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:29.678628 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:29.679621 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:29.679621 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:29.679621 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:29.679621 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:29.679621 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:29.679621 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:29.679621 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:29.679621 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:29.679621 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:29.679621 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:29.679621 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:29.679621 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:29.679621 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:29.679621 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:29.679621 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:29.679621 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:29.679621 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:29.680432 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:29.680432 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:29.680432 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:29.680432 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 268/548] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:29.685643 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52709ed-5e92-49cd-81b5-1fe5a1801f80 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:29.786717 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8142b1-c103-4337-babe-b1b136275114 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:29.787736 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8142b1-c103-4337-babe-b1b136275114 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:29.788226 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8142b1-c103-4337-babe-b1b136275114 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:29.790042 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 269/549] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 00:48:29.794127 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:29.796670 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba80525-626d-4ea7-b613-b3c7da2aaad8 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:29.797171 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba80525-626d-4ea7-b613-b3c7da2aaad8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:29.797668 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba80525-626d-4ea7-b613-b3c7da2aaad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:29.817273 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1ba80525-626d-4ea7-b613-b3c7da2aaad8 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:29.818804 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1ba80525-626d-4ea7-b613-b3c7da2aaad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:29.869035 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52709ed-5e92-49cd-81b5-1fe5a1801f80 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:29.893982 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 270/550] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:29.900489 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba80525-626d-4ea7-b613-b3c7da2aaad8 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:29.918517 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61104c6e-86c2-4e33-931f-e0eab0bf9217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:29.919450 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61104c6e-86c2-4e33-931f-e0eab0bf9217 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:29.923859 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61104c6e-86c2-4e33-931f-e0eab0bf9217 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:29.923859 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61104c6e-86c2-4e33-931f-e0eab0bf9217 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:29.923859 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61104c6e-86c2-4e33-931f-e0eab0bf9217 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:29.923859 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61104c6e-86c2-4e33-931f-e0eab0bf9217 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:29.923859 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61104c6e-86c2-4e33-931f-e0eab0bf9217 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:29.924758 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:29.950925 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 281/551] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:29.956370 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61104c6e-86c2-4e33-931f-e0eab0bf9217 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:30.042785 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba80525-626d-4ea7-b613-b3c7da2aaad8 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:30.109972 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61104c6e-86c2-4e33-931f-e0eab0bf9217 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:30.112613 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54123104-935f-40c7-9a02-7f785194e007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:30.113254 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54123104-935f-40c7-9a02-7f785194e007 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:30.113533 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54123104-935f-40c7-9a02-7f785194e007 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:30.113808 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54123104-935f-40c7-9a02-7f785194e007 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:30.132646 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54123104-935f-40c7-9a02-7f785194e007 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:30.141365 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-54123104-935f-40c7-9a02-7f785194e007 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:30.141365 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:30.141365 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:30.141365 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:30.141365 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:30.141365 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:30.141365 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:30.141365 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:30.141365 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:30.141365 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:30.141365 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:30.141365 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:30.141365 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:30.141365 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:30.141365 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:30.141365 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:30.142276 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:30.142276 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:30.142276 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:30.142276 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:30.142276 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:30.142276 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:30.142276 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:30.142276 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:30.142276 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:30.142276 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:30.142276 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:30.142276 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:30.142276 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:30.142276 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:30.142276 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:30.142276 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:30.143633 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:30.143633 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:30.143633 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:30.143633 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:30.143633 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:30.143633 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:30.143633 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:30.143633 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:30.143633 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:30.143633 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:30.143633 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:30.143633 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:30.143633 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 271/552] 10.222.0.27 () {56 vars in 1102 bytes} [Sun Sep 4 00:48:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:30.151477 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54123104-935f-40c7-9a02-7f785194e007 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:30.162815 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15384c84-0824-4cfc-80d7-35e8f521aaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:30.163537 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15384c84-0824-4cfc-80d7-35e8f521aaef None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:30.164141 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15384c84-0824-4cfc-80d7-35e8f521aaef None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:30.164526 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:30.171795 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15384c84-0824-4cfc-80d7-35e8f521aaef None admin] PATH_INFO: `/v3/users/swift` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:30.185255 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15384c84-0824-4cfc-80d7-35e8f521aaef None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:30.187004 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15384c84-0824-4cfc-80d7-35e8f521aaef None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:30.212170 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-15384c84-0824-4cfc-80d7-35e8f521aaef None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 04 00:48:30.212170 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:30.212170 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:30.212170 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:30.212170 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:30.212170 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:30.212170 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:30.212170 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:30.212170 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:30.212170 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:30.212170 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:30.212170 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:30.212170 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:48:30.212170 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:48:30.212170 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:48:30.212170 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:48:30.213168 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:30.213168 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:30.213168 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:48:30.213168 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:30.213168 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:48:30.213168 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:30.213168 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:30.213168 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:30.213168 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:30.213168 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:30.213168 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:30.213168 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:30.213168 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:30.213168 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:30.213168 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:30.213168 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:30.213168 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:30.213168 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:30.214074 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:30.214074 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:30.214074 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:48:30.214074 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:48:30.214074 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:48:30.214074 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:48:30.214074 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:48:30.214074 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:48:30.214074 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 04 00:48:30.214074 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:30.214074 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 282/553] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:48:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:48:30.221712 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15384c84-0824-4cfc-80d7-35e8f521aaef None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:30.262265 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:30.278718 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54123104-935f-40c7-9a02-7f785194e007 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:30.326481 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6a571-77ab-474b-9600-bd10183bcbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:30.327379 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6a571-77ab-474b-9600-bd10183bcbf9 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:30.327669 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6a571-77ab-474b-9600-bd10183bcbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:30.328200 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6a571-77ab-474b-9600-bd10183bcbf9 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:30.328894 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af6a571-77ab-474b-9600-bd10183bcbf9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:30.329193 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af6a571-77ab-474b-9600-bd10183bcbf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:30.331169 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af6a571-77ab-474b-9600-bd10183bcbf9 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:30.342073 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 272/554] 10.222.0.27 () {56 vars in 1103 bytes} [Sun Sep 4 00:48:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:30.342921 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6a571-77ab-474b-9600-bd10183bcbf9 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:30.371829 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15384c84-0824-4cfc-80d7-35e8f521aaef None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:30.407091 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa0a63-31fb-4f57-8936-fa6d502076c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:30.407614 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa0a63-31fb-4f57-8936-fa6d502076c5 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:30.407895 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa0a63-31fb-4f57-8936-fa6d502076c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:30.408170 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa0a63-31fb-4f57-8936-fa6d502076c5 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:30.409756 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aa0a63-31fb-4f57-8936-fa6d502076c5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:30.410061 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aa0a63-31fb-4f57-8936-fa6d502076c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:30.411526 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aa0a63-31fb-4f57-8936-fa6d502076c5 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:30.437001 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 283/555] 10.222.0.27 () {56 vars in 1135 bytes} [Sun Sep 4 00:48:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:30.437777 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa0a63-31fb-4f57-8936-fa6d502076c5 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:30.463968 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6a571-77ab-474b-9600-bd10183bcbf9 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:30.487174 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:30.502128 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0c9ce-120f-4043-9aaa-060d9b7f0635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:30.503369 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0c9ce-120f-4043-9aaa-060d9b7f0635 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:30.503664 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0c9ce-120f-4043-9aaa-060d9b7f0635 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:30.503943 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0c9ce-120f-4043-9aaa-060d9b7f0635 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:30.514864 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0c9ce-120f-4043-9aaa-060d9b7f0635 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:30.520901 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-2da0c9ce-120f-4043-9aaa-060d9b7f0635 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:30.520901 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:30.520901 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:30.520901 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:30.520901 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:30.520901 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:30.520901 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:30.520901 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:30.520901 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:30.520901 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:30.520901 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:30.520901 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:30.520901 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:30.520901 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:30.520901 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:30.520901 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:30.522734 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:30.522734 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:30.522734 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:30.522734 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:30.522734 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:30.522734 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:30.522734 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:30.522734 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:30.522734 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:30.522734 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:30.522734 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:30.522734 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:30.522734 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:30.522734 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:30.522734 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:30.522734 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:30.524792 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:30.524792 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:30.524792 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:30.524792 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:30.524792 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:30.524792 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:30.524792 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:30.524792 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:30.524792 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:30.524792 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:30.524792 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:30.524792 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:30.524792 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 273/556] 10.222.0.27 () {56 vars in 1102 bytes} [Sun Sep 4 00:48:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:30.524792 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0c9ce-120f-4043-9aaa-060d9b7f0635 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:30.537629 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:30.547827 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa0a63-31fb-4f57-8936-fa6d502076c5 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:30.584785 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918f510-db92-467e-b412-e642b978f9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:30.585457 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918f510-db92-467e-b412-e642b978f9bb None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:30.585741 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918f510-db92-467e-b412-e642b978f9bb None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:30.586011 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918f510-db92-467e-b412-e642b978f9bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:30.597686 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0918f510-db92-467e-b412-e642b978f9bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:30.599336 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0918f510-db92-467e-b412-e642b978f9bb None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:30.615490 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-0918f510-db92-467e-b412-e642b978f9bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:30.615490 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:30.615490 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:30.615490 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:30.615490 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:30.615490 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:30.615490 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:30.615490 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:30.615490 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:30.615490 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:30.615490 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:30.615490 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:30.615490 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:30.615490 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:30.615490 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:30.615490 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:30.618594 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:30.618594 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:30.618594 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:30.618594 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:30.618594 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:30.618594 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:30.618594 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:30.618594 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:30.618594 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:30.618594 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:30.618594 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:30.618594 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:30.618594 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:30.618594 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:30.618594 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:30.618594 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:30.618594 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:30.620728 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:30.620728 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:30.620728 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:30.620728 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 284/557] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:30.620728 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918f510-db92-467e-b412-e642b978f9bb None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:30.644387 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0c9ce-120f-4043-9aaa-060d9b7f0635 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:30.683951 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de07eda-19c4-404e-b25b-e88a758de7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:30.684612 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de07eda-19c4-404e-b25b-e88a758de7a4 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:30.684900 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de07eda-19c4-404e-b25b-e88a758de7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:30.685177 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de07eda-19c4-404e-b25b-e88a758de7a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:30.721828 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:30.728402 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918f510-db92-467e-b412-e642b978f9bb None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:30.762816 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de07eda-19c4-404e-b25b-e88a758de7a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:30.764751 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de07eda-19c4-404e-b25b-e88a758de7a4 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:30.766400 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b625c33-4826-4857-9021-f4baa8c1f9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:30.767260 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b625c33-4826-4857-9021-f4baa8c1f9c7 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:30.768355 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b625c33-4826-4857-9021-f4baa8c1f9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:30.768806 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b625c33-4826-4857-9021-f4baa8c1f9c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:30.769727 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b625c33-4826-4857-9021-f4baa8c1f9c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:30.770265 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b625c33-4826-4857-9021-f4baa8c1f9c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:30.771960 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b625c33-4826-4857-9021-f4baa8c1f9c7 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:30.778281 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:30.782079 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 285/558] 10.222.0.27 () {56 vars in 1103 bytes} [Sun Sep 4 00:48:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:30.783319 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b625c33-4826-4857-9021-f4baa8c1f9c7 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:30.883820 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 274/559] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:48:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:48:30.883820 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de07eda-19c4-404e-b25b-e88a758de7a4 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:30.899754 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b625c33-4826-4857-9021-f4baa8c1f9c7 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:30.937035 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ccb0ce-178d-4201-9cf0-e28c959b9099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:30.937553 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ccb0ce-178d-4201-9cf0-e28c959b9099 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:30.937831 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ccb0ce-178d-4201-9cf0-e28c959b9099 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:30.938102 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ccb0ce-178d-4201-9cf0-e28c959b9099 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:30.938741 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ccb0ce-178d-4201-9cf0-e28c959b9099 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:30.939035 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ccb0ce-178d-4201-9cf0-e28c959b9099 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:30.940477 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ccb0ce-178d-4201-9cf0-e28c959b9099 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:30.956739 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 286/560] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:30.956739 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ccb0ce-178d-4201-9cf0-e28c959b9099 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:30.978384 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:30.996743 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de07eda-19c4-404e-b25b-e88a758de7a4 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:31.006447 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:31.031654 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834fc30-e83d-4ee8-bf9f-30a1be763897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:31.032382 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834fc30-e83d-4ee8-bf9f-30a1be763897 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:31.032695 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834fc30-e83d-4ee8-bf9f-30a1be763897 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:31.032966 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834fc30-e83d-4ee8-bf9f-30a1be763897 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:31.044435 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3834fc30-e83d-4ee8-bf9f-30a1be763897 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:31.046432 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3834fc30-e83d-4ee8-bf9f-30a1be763897 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:31.062070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-3834fc30-e83d-4ee8-bf9f-30a1be763897 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:31.062070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:31.062070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:31.062070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:31.062070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:31.062070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:31.062070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:31.062070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:31.062070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:31.062070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:31.062070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:31.062070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:31.062070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:31.062070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:31.062070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:31.062070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:31.063026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:31.063026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:31.063026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:31.063026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:31.063026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:31.063026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:31.063026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:31.063026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:31.063026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:31.063026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:31.063026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:31.063026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:31.063026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:31.063026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:31.063026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:31.063026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:31.063026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:31.063868 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:31.063868 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:31.063868 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:31.063868 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 275/561] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:31.065103 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2101ed5-2141-41b0-a657-6eec1f14135b None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:31.065205 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2101ed5-2141-41b0-a657-6eec1f14135b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:31.065531 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2101ed5-2141-41b0-a657-6eec1f14135b None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:31.071005 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 276/562] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:31.071005 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2101ed5-2141-41b0-a657-6eec1f14135b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:31.073609 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ccb0ce-178d-4201-9cf0-e28c959b9099 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:31.109742 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d00a03-1026-4402-ac7a-dc54296b022b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:31.110365 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d00a03-1026-4402-ac7a-dc54296b022b None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:31.110643 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d00a03-1026-4402-ac7a-dc54296b022b None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:31.110910 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d00a03-1026-4402-ac7a-dc54296b022b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:31.121032 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d00a03-1026-4402-ac7a-dc54296b022b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:31.122478 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d00a03-1026-4402-ac7a-dc54296b022b None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:31.137381 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-93d00a03-1026-4402-ac7a-dc54296b022b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:31.137381 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:31.137381 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:31.137381 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:31.137381 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:31.137381 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:31.137381 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:31.137381 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:31.137381 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:31.137381 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:31.137381 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:31.137381 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:31.137381 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:31.137381 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:31.137381 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:31.137381 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:31.138306 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:31.138306 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:31.138306 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:31.138306 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:31.138306 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:31.138306 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:31.138306 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:31.138306 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:31.138306 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:31.138306 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:31.138306 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:31.138306 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:31.138306 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:31.138306 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:31.138306 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:31.138306 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:31.138306 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:31.139148 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:31.139148 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:31.139148 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:31.139148 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 287/563] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:31.139375 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d00a03-1026-4402-ac7a-dc54296b022b None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:31.188277 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2101ed5-2141-41b0-a657-6eec1f14135b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:31.215066 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:31.229118 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f285bbc0-fb9f-43a3-9b26-da86386beff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:31.237063 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285bbc0-fb9f-43a3-9b26-da86386beff8 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:31.237063 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285bbc0-fb9f-43a3-9b26-da86386beff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:31.237063 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285bbc0-fb9f-43a3-9b26-da86386beff8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:31.238942 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f285bbc0-fb9f-43a3-9b26-da86386beff8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:31.240367 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f285bbc0-fb9f-43a3-9b26-da86386beff8 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:31.250039 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d00a03-1026-4402-ac7a-dc54296b022b None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:31.251839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-f285bbc0-fb9f-43a3-9b26-da86386beff8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 04 00:48:31.251839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:31.251839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:31.251839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:31.251839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:31.251839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:31.251839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:31.251839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:31.251839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:31.251839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:31.251839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:31.251839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:31.251839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:48:31.251839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:48:31.251839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:48:31.251839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:48:31.252890 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:31.252890 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:31.252890 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:48:31.252890 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:31.252890 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:48:31.252890 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:31.252890 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:31.252890 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:31.252890 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:31.252890 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:31.252890 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:31.252890 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:31.252890 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:31.252890 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:31.252890 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:31.252890 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:31.252890 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:31.252890 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:31.253841 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:31.253841 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:31.253841 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:48:31.253841 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:48:31.253841 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:48:31.253841 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:48:31.253841 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:48:31.253841 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:48:31.253841 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 04 00:48:31.253841 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:31.253841 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 277/564] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:48:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:31.254446 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda04031-c3d4-470d-9952-ec3a367b83cb None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:31.254515 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda04031-c3d4-470d-9952-ec3a367b83cb None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:31.255062 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda04031-c3d4-470d-9952-ec3a367b83cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:31.272895 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-eda04031-c3d4-470d-9952-ec3a367b83cb None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:31.274115 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-eda04031-c3d4-470d-9952-ec3a367b83cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:31.288632 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:31.294188 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b95b22-b28d-4a04-a790-7204c5479625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:31.295019 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b95b22-b28d-4a04-a790-7204c5479625 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:31.295306 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b95b22-b28d-4a04-a790-7204c5479625 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:31.295583 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b95b22-b28d-4a04-a790-7204c5479625 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:31.296221 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b95b22-b28d-4a04-a790-7204c5479625 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:31.296557 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b95b22-b28d-4a04-a790-7204c5479625 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:31.297977 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b95b22-b28d-4a04-a790-7204c5479625 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:31.315316 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 288/565] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:31.316238 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b95b22-b28d-4a04-a790-7204c5479625 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:31.342358 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 278/566] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:31.343203 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda04031-c3d4-470d-9952-ec3a367b83cb None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:31.417443 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b95b22-b28d-4a04-a790-7204c5479625 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:31.446036 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:31.451848 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed68bf-9deb-45d2-8d8c-07e5612556ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:31.452469 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed68bf-9deb-45d2-8d8c-07e5612556ea None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:31.452762 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed68bf-9deb-45d2-8d8c-07e5612556ea None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:31.452860 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda04031-c3d4-470d-9952-ec3a367b83cb None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:31.453038 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed68bf-9deb-45d2-8d8c-07e5612556ea None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:31.453681 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ed68bf-9deb-45d2-8d8c-07e5612556ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:31.453985 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ed68bf-9deb-45d2-8d8c-07e5612556ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:31.455903 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ed68bf-9deb-45d2-8d8c-07e5612556ea None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:31.472075 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 289/567] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:31.472075 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed68bf-9deb-45d2-8d8c-07e5612556ea None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:31.491585 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f80db-24d7-4b3f-96b9-9f6f49022f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:31.492196 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f80db-24d7-4b3f-96b9-9f6f49022f5c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:31.492497 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f80db-24d7-4b3f-96b9-9f6f49022f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:31.492780 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f80db-24d7-4b3f-96b9-9f6f49022f5c None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:31.493506 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f80db-24d7-4b3f-96b9-9f6f49022f5c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:31.493813 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f80db-24d7-4b3f-96b9-9f6f49022f5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:31.495247 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f80db-24d7-4b3f-96b9-9f6f49022f5c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:31.516352 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 279/568] 10.222.0.27 () {56 vars in 1137 bytes} [Sun Sep 4 00:48:31 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:31.516352 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32aa92-47aa-4faf-83b5-942387850120 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:31.516352 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32aa92-47aa-4faf-83b5-942387850120 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:31.516352 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32aa92-47aa-4faf-83b5-942387850120 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:31.516352 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 280/569] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:31.516352 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32aa92-47aa-4faf-83b5-942387850120 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:31.528409 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:31.589186 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed68bf-9deb-45d2-8d8c-07e5612556ea None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:31.623339 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32aa92-47aa-4faf-83b5-942387850120 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:31.626249 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be2c8a-de32-4d2f-a723-f77bc165ff03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:31.626877 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be2c8a-de32-4d2f-a723-f77bc165ff03 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:31.627708 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be2c8a-de32-4d2f-a723-f77bc165ff03 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:31.627981 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be2c8a-de32-4d2f-a723-f77bc165ff03 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:31.635350 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6be2c8a-de32-4d2f-a723-f77bc165ff03 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:31.636813 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6be2c8a-de32-4d2f-a723-f77bc165ff03 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:31.648501 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-e6be2c8a-de32-4d2f-a723-f77bc165ff03 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 04 00:48:31.648501 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:31.648501 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:31.648501 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:31.648501 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:31.648501 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:31.648501 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:31.648501 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:31.648501 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:31.648501 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:31.648501 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:31.648501 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:31.648501 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:48:31.648501 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:48:31.648501 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:48:31.648501 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:48:31.649467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:31.649467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:31.649467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:48:31.649467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:31.649467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:48:31.649467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:31.649467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:31.649467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:31.649467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:31.649467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:31.649467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:31.649467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:31.649467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:31.649467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:31.649467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:31.649467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:31.649467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:31.649467 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:31.650411 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:31.650411 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:31.650411 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:48:31.650411 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:48:31.650411 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:48:31.650411 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:48:31.650411 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:48:31.650411 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:48:31.650411 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 04 00:48:31.650411 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:31.650411 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 290/570] 10.222.0.27 () {56 vars in 1093 bytes} [Sun Sep 4 00:48:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:31.650993 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1b09d-15d5-40d7-8fa6-66e4c0d80f95 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:31.650993 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1b09d-15d5-40d7-8fa6-66e4c0d80f95 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:31.651300 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1b09d-15d5-40d7-8fa6-66e4c0d80f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:31.662304 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce59452-e3ff-4efc-8d33-002830951d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:31.662925 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce59452-e3ff-4efc-8d33-002830951d3e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:31.663200 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce59452-e3ff-4efc-8d33-002830951d3e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:31.663468 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce59452-e3ff-4efc-8d33-002830951d3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:31.670356 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce59452-e3ff-4efc-8d33-002830951d3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:31.670771 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-91b1b09d-15d5-40d7-8fa6-66e4c0d80f95 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:31.671771 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce59452-e3ff-4efc-8d33-002830951d3e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:31.671972 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-91b1b09d-15d5-40d7-8fa6-66e4c0d80f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:31.682902 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-8ce59452-e3ff-4efc-8d33-002830951d3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:31.682902 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:31.682902 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:31.682902 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:31.682902 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:31.682902 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:31.682902 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:31.682902 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:31.682902 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:31.682902 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:31.682902 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:31.682902 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:31.682902 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:48:31.682902 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:48:31.682902 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:48:31.683936 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:48:31.683936 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:31.683936 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:31.683936 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:31.683936 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:31.683936 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:31.683936 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:31.683936 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:31.683936 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:31.683936 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:31.683936 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:31.683936 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:31.683936 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:31.683936 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:31.683936 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:31.683936 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:31.683936 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:31.691084 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:48:31.691084 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:48:31.691084 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:48:31.691084 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:48:31.691084 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:48:31.691084 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:48:31.691084 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:31.691084 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:31.691084 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 281/571] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:48:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:31.691084 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce59452-e3ff-4efc-8d33-002830951d3e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:31.719037 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:31.739011 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 291/572] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:31.739843 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1b09d-15d5-40d7-8fa6-66e4c0d80f95 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:31.749476 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:31.790643 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce59452-e3ff-4efc-8d33-002830951d3e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:31.827599 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce3a4f-a27f-4182-acbc-3ef6de85df30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:31.828346 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce3a4f-a27f-4182-acbc-3ef6de85df30 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:31.828717 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce3a4f-a27f-4182-acbc-3ef6de85df30 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:31.829053 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce3a4f-a27f-4182-acbc-3ef6de85df30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:31.840364 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ce3a4f-a27f-4182-acbc-3ef6de85df30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:31.841893 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ce3a4f-a27f-4182-acbc-3ef6de85df30 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:31.856076 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1b09d-15d5-40d7-8fa6-66e4c0d80f95 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:31.857593 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-e8ce3a4f-a27f-4182-acbc-3ef6de85df30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:31.857593 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:31.857593 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:31.857593 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:31.857593 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:31.857593 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:31.857593 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:31.857593 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:31.857593 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:31.857593 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:31.857593 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:31.857593 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:31.857593 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:31.857593 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:31.857593 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:31.857593 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:31.858707 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:31.858707 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:31.858707 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:31.858707 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:31.858707 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:31.858707 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:31.858707 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:31.858707 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:31.858707 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:31.858707 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:31.858707 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:31.858707 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:31.858707 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:31.858707 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:31.858707 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:31.858707 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:31.858707 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:31.859777 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:31.859777 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:31.859777 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:31.864217 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 282/573] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:31.865309 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce3a4f-a27f-4182-acbc-3ef6de85df30 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:31.894490 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7aa3a6-43a7-43bd-9e45-00674c4acf92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:31.895153 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7aa3a6-43a7-43bd-9e45-00674c4acf92 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:31.895452 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7aa3a6-43a7-43bd-9e45-00674c4acf92 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:31.895749 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7aa3a6-43a7-43bd-9e45-00674c4acf92 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:31.905976 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7aa3a6-43a7-43bd-9e45-00674c4acf92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:31.911552 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-6f7aa3a6-43a7-43bd-9e45-00674c4acf92 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:31.911552 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:31.911552 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:31.911552 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:31.911552 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:31.911552 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:31.911552 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:31.911552 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:31.911552 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:31.911552 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:31.911552 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:31.911552 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:31.911552 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:31.911552 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:31.911552 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:31.911552 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:31.912452 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:31.912452 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:31.912452 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:31.912452 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:31.912452 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:31.912452 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:31.912452 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:31.912452 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:31.912452 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:31.912452 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:31.912452 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:31.912452 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:31.912452 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:31.912452 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:31.912452 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:31.912452 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:31.913291 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:31.913291 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:31.913291 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:31.913291 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:31.913291 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:31.913291 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:31.913291 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:31.913291 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:31.913291 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:31.913291 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:31.913291 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:31.913291 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:31.913908 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 292/574] 10.222.0.27 () {56 vars in 1102 bytes} [Sun Sep 4 00:48:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:31.914454 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7aa3a6-43a7-43bd-9e45-00674c4acf92 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:31.960563 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:31.979295 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce3a4f-a27f-4182-acbc-3ef6de85df30 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:32.001525 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:32.016918 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14875ea9-c00f-4087-84a9-d0f8e8176198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:32.017525 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14875ea9-c00f-4087-84a9-d0f8e8176198 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:32.017860 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14875ea9-c00f-4087-84a9-d0f8e8176198 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:32.018139 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14875ea9-c00f-4087-84a9-d0f8e8176198 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:32.018973 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14875ea9-c00f-4087-84a9-d0f8e8176198 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:32.019280 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14875ea9-c00f-4087-84a9-d0f8e8176198 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:32.020741 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14875ea9-c00f-4087-84a9-d0f8e8176198 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:32.022937 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7aa3a6-43a7-43bd-9e45-00674c4acf92 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:32.038195 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 283/575] 10.222.0.27 () {56 vars in 1133 bytes} [Sun Sep 4 00:48:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:32.039018 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14875ea9-c00f-4087-84a9-d0f8e8176198 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:32.060197 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47324e29-2407-4c23-911a-04fc69df163a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:32.060821 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47324e29-2407-4c23-911a-04fc69df163a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:32.061102 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47324e29-2407-4c23-911a-04fc69df163a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:32.061375 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47324e29-2407-4c23-911a-04fc69df163a None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:32.062057 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47324e29-2407-4c23-911a-04fc69df163a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:32.062357 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47324e29-2407-4c23-911a-04fc69df163a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:32.063777 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47324e29-2407-4c23-911a-04fc69df163a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:32.075573 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 293/576] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:48:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:32.076349 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47324e29-2407-4c23-911a-04fc69df163a None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:32.136086 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14875ea9-c00f-4087-84a9-d0f8e8176198 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:32.168556 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ace0be-d253-4f6e-9a32-121a8561c55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:32.169148 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ace0be-d253-4f6e-9a32-121a8561c55e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:32.169421 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ace0be-d253-4f6e-9a32-121a8561c55e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:32.169690 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ace0be-d253-4f6e-9a32-121a8561c55e None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:32.170315 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ace0be-d253-4f6e-9a32-121a8561c55e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:32.170609 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ace0be-d253-4f6e-9a32-121a8561c55e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:32.171964 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ace0be-d253-4f6e-9a32-121a8561c55e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:32.176681 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:32.179564 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47324e29-2407-4c23-911a-04fc69df163a None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:32.183282 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 284/577] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:32.184069 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ace0be-d253-4f6e-9a32-121a8561c55e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:32.214556 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace33104-bc44-405e-ac15-6bcf73d09226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:32.215157 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace33104-bc44-405e-ac15-6bcf73d09226 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:32.215435 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace33104-bc44-405e-ac15-6bcf73d09226 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:32.215705 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace33104-bc44-405e-ac15-6bcf73d09226 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:32.216364 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace33104-bc44-405e-ac15-6bcf73d09226 None admin] RBAC: Adding query filter params (name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:32.216690 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace33104-bc44-405e-ac15-6bcf73d09226 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:32.218044 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace33104-bc44-405e-ac15-6bcf73d09226 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:32.227397 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 294/578] 10.222.0.27 () {56 vars in 1103 bytes} [Sun Sep 4 00:48:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:32.227397 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace33104-bc44-405e-ac15-6bcf73d09226 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:32.229720 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:32.290690 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ace0be-d253-4f6e-9a32-121a8561c55e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:32.324927 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c55c6d-c400-452f-a783-33ed8bdb0998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:32.325538 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c55c6d-c400-452f-a783-33ed8bdb0998 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:32.325818 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c55c6d-c400-452f-a783-33ed8bdb0998 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:32.326093 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c55c6d-c400-452f-a783-33ed8bdb0998 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:32.326361 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace33104-bc44-405e-ac15-6bcf73d09226 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:32.336253 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c55c6d-c400-452f-a783-33ed8bdb0998 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:32.341424 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-12c55c6d-c400-452f-a783-33ed8bdb0998 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:32.341424 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:32.341424 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:32.341424 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:32.341424 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:32.341424 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:32.341424 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:32.341424 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:32.341424 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:32.341424 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:32.341424 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:32.341424 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:32.341424 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:32.341424 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:32.341424 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:32.341424 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:32.342435 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:32.342435 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:32.342435 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:32.342435 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:32.342435 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:32.342435 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:32.342435 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:32.342435 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:32.342435 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:32.342435 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:32.342435 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:32.342435 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:32.342435 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:32.342435 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:32.342435 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:32.342435 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:32.343277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:32.343277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:32.343277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:32.343277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:32.343277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:32.343277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:32.343277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:32.343277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:32.343277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:32.343277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:32.343277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:32.343277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:32.343277 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 285/579] 10.222.0.27 () {56 vars in 1102 bytes} [Sun Sep 4 00:48:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:32.343277 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c55c6d-c400-452f-a783-33ed8bdb0998 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:32.362894 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1fbac2-48da-490c-9535-2020417a6c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:32.363522 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fbac2-48da-490c-9535-2020417a6c7a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:32.363804 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fbac2-48da-490c-9535-2020417a6c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:32.364075 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fbac2-48da-490c-9535-2020417a6c7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:32.374297 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1fbac2-48da-490c-9535-2020417a6c7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:32.375745 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1fbac2-48da-490c-9535-2020417a6c7a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:32.389572 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-da1fbac2-48da-490c-9535-2020417a6c7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:32.389572 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:32.389572 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:32.389572 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:32.389572 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:32.389572 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:32.389572 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:32.389572 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:32.389572 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:32.389572 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:32.389572 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:32.389572 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:32.389572 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:32.389572 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:32.389572 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:32.389572 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:32.390557 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:32.390557 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:32.390557 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:32.390557 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:32.390557 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:32.390557 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:32.390557 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:32.390557 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:32.390557 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:32.390557 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:32.390557 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:32.390557 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:32.390557 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:32.390557 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:32.390557 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:32.390557 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:32.390557 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:32.391503 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:32.391503 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:32.391503 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:32.391503 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 295/580] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:32.391503 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1fbac2-48da-490c-9535-2020417a6c7a None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:32.406497 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:32.444131 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:32.447810 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c55c6d-c400-452f-a783-33ed8bdb0998 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:32.481397 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd63a7c-4b2a-47de-bdc5-0f06a75d1359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:32.481932 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd63a7c-4b2a-47de-bdc5-0f06a75d1359 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:32.482212 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd63a7c-4b2a-47de-bdc5-0f06a75d1359 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:32.482484 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd63a7c-4b2a-47de-bdc5-0f06a75d1359 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:32.483199 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd63a7c-4b2a-47de-bdc5-0f06a75d1359 None admin] RBAC: Adding query filter params (role.id=6e3e398ba29e436e8532a45d2fdf1321, scope.project.id=5100a8d316d243fb8f82e9044e1b3b06, user.id=2297d8b3fe8e47eb9fd39a6e3eea8f54) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:32.483506 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd63a7c-4b2a-47de-bdc5-0f06a75d1359 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:32.484969 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd63a7c-4b2a-47de-bdc5-0f06a75d1359 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:32.490721 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 286/581] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:48:32 2022] GET /identity/v3/role_assignments?user.id=2297d8b3fe8e47eb9fd39a6e3eea8f54&scope.project.id=5100a8d316d243fb8f82e9044e1b3b06&role.id=6e3e398ba29e436e8532a45d2fdf1321 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:32.490949 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1fbac2-48da-490c-9535-2020417a6c7a None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:32.491506 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd63a7c-4b2a-47de-bdc5-0f06a75d1359 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:32.546331 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa75d2-bffb-41fd-b1e2-5475ee1dcf17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:32.547219 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa75d2-bffb-41fd-b1e2-5475ee1dcf17 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:32.547219 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa75d2-bffb-41fd-b1e2-5475ee1dcf17 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:32.547513 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa75d2-bffb-41fd-b1e2-5475ee1dcf17 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:32.556261 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa75d2-bffb-41fd-b1e2-5475ee1dcf17 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:32.557783 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa75d2-bffb-41fd-b1e2-5475ee1dcf17 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:32.570152 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-71aa75d2-bffb-41fd-b1e2-5475ee1dcf17 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:32.570152 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:32.570152 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:32.570152 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:32.570152 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:32.570152 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:32.570152 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:32.570152 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:32.570152 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:32.570152 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:32.570152 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:32.570152 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:32.570152 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:48:32.570152 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:48:32.570152 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:48:32.571086 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:48:32.571086 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:32.571086 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:32.571086 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:32.571086 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:32.571086 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:32.571086 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:32.571086 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:32.571086 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:32.571086 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:32.571086 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:32.571086 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:32.571086 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:32.571086 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:32.571086 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:32.571086 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:32.571086 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:32.571948 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:48:32.571948 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:48:32.571948 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:48:32.571948 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:48:32.571948 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:48:32.571948 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:48:32.571948 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:32.571948 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:32.571948 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 296/582] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:48:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:32.573199 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa75d2-bffb-41fd-b1e2-5475ee1dcf17 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:32.610144 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd63a7c-4b2a-47de-bdc5-0f06a75d1359 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:32.649080 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d9ca59-267e-4e45-bda6-be6f13484ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:32.649707 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9ca59-267e-4e45-bda6-be6f13484ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:32.649984 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9ca59-267e-4e45-bda6-be6f13484ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:32.650254 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9ca59-267e-4e45-bda6-be6f13484ec7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:32.657564 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:32.662216 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d9ca59-267e-4e45-bda6-be6f13484ec7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:32.663649 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d9ca59-267e-4e45-bda6-be6f13484ec7 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:32.679095 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-22d9ca59-267e-4e45-bda6-be6f13484ec7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:32.679095 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:32.679095 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:32.679095 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:32.679095 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:32.679095 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:32.679095 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:32.679095 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:32.679095 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:32.679095 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:32.679095 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:32.679095 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:32.679095 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:32.679095 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:32.679095 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:32.679095 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:32.680240 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:32.680240 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:32.680240 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:32.680240 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:32.680240 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:32.680240 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:32.680240 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:32.680240 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:32.680240 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:32.680240 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:32.680240 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:32.680240 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:32.680240 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:32.680240 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:32.680240 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:32.680240 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:32.680240 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:32.681506 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:32.681506 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:32.681506 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:32.684826 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 287/583] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:32.685677 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d9ca59-267e-4e45-bda6-be6f13484ec7 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:32.687370 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa75d2-bffb-41fd-b1e2-5475ee1dcf17 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:32.702853 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:32.725124 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea07f8-418f-478f-b66d-2b4480ae529f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:32.725730 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea07f8-418f-478f-b66d-2b4480ae529f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:32.726005 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea07f8-418f-478f-b66d-2b4480ae529f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:32.726273 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea07f8-418f-478f-b66d-2b4480ae529f None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:32.726934 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ea07f8-418f-478f-b66d-2b4480ae529f None admin] RBAC: Adding query filter params (name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:32.727238 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ea07f8-418f-478f-b66d-2b4480ae529f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:32.728683 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ea07f8-418f-478f-b66d-2b4480ae529f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:32.743639 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 297/584] 10.222.0.27 () {56 vars in 1103 bytes} [Sun Sep 4 00:48:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:32.744580 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea07f8-418f-478f-b66d-2b4480ae529f None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:32.794320 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d9ca59-267e-4e45-bda6-be6f13484ec7 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:32.832486 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6240d1-289d-4340-a134-9aab6ed5c51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:32.833116 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6240d1-289d-4340-a134-9aab6ed5c51c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:32.833397 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6240d1-289d-4340-a134-9aab6ed5c51c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:32.833671 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6240d1-289d-4340-a134-9aab6ed5c51c None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:32.834306 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6240d1-289d-4340-a134-9aab6ed5c51c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:32.834607 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6240d1-289d-4340-a134-9aab6ed5c51c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:32.835961 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6240d1-289d-4340-a134-9aab6ed5c51c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:32.848368 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 288/585] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:32.849200 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6240d1-289d-4340-a134-9aab6ed5c51c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:32.860346 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea07f8-418f-478f-b66d-2b4480ae529f None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:32.898911 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b79476-8ed3-4193-89b9-31ec6c39b609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:32.899514 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b79476-8ed3-4193-89b9-31ec6c39b609 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:32.899800 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b79476-8ed3-4193-89b9-31ec6c39b609 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:32.900072 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b79476-8ed3-4193-89b9-31ec6c39b609 None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:32.900801 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b79476-8ed3-4193-89b9-31ec6c39b609 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:32.901095 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b79476-8ed3-4193-89b9-31ec6c39b609 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:32.902571 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b79476-8ed3-4193-89b9-31ec6c39b609 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:32.905559 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:32.916543 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 298/586] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:48:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:32.917370 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b79476-8ed3-4193-89b9-31ec6c39b609 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:32.941826 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:32.967257 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6240d1-289d-4340-a134-9aab6ed5c51c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:33.007393 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e79112-1a72-40b0-8406-ff2e6ca9d004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:33.008013 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e79112-1a72-40b0-8406-ff2e6ca9d004 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:33.008219 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e79112-1a72-40b0-8406-ff2e6ca9d004 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:33.008451 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e79112-1a72-40b0-8406-ff2e6ca9d004 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:33.009109 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e79112-1a72-40b0-8406-ff2e6ca9d004 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:33.009417 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e79112-1a72-40b0-8406-ff2e6ca9d004 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:33.010780 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e79112-1a72-40b0-8406-ff2e6ca9d004 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:33.024847 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 289/587] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:33.025651 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e79112-1a72-40b0-8406-ff2e6ca9d004 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:33.047559 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b79476-8ed3-4193-89b9-31ec6c39b609 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:33.085392 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4125bb2-6eab-4223-a0ea-cd930ff304b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:33.086293 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4125bb2-6eab-4223-a0ea-cd930ff304b5 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:33.086570 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4125bb2-6eab-4223-a0ea-cd930ff304b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:33.086838 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4125bb2-6eab-4223-a0ea-cd930ff304b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:33.097755 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4125bb2-6eab-4223-a0ea-cd930ff304b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:33.099190 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4125bb2-6eab-4223-a0ea-cd930ff304b5 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:33.113743 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-f4125bb2-6eab-4223-a0ea-cd930ff304b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:33.113743 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:33.113743 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:33.113743 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:33.113743 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:33.113743 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:33.113743 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:33.113743 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:33.113743 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:33.113743 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:33.113743 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:33.113743 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:33.113743 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:33.113743 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:33.113743 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:33.113743 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:33.114882 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:33.114882 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:33.114882 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:33.114882 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:33.114882 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:33.114882 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:33.114882 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:33.114882 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:33.114882 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:33.114882 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:33.114882 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:33.114882 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:33.114882 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:33.114882 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:33.114882 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:33.114882 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:33.114882 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:33.115991 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:33.115991 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:33.115991 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:33.119528 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 299/588] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:33.120339 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4125bb2-6eab-4223-a0ea-cd930ff304b5 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:33.136227 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e79112-1a72-40b0-8406-ff2e6ca9d004 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:33.171266 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:33.173926 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07592070-4801-4c9b-84c7-01f72a5dbab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:33.174722 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07592070-4801-4c9b-84c7-01f72a5dbab4 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:33.175007 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07592070-4801-4c9b-84c7-01f72a5dbab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:33.175287 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07592070-4801-4c9b-84c7-01f72a5dbab4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:33.177338 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:33.183497 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07592070-4801-4c9b-84c7-01f72a5dbab4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:33.184966 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07592070-4801-4c9b-84c7-01f72a5dbab4 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:33.196771 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-07592070-4801-4c9b-84c7-01f72a5dbab4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:33.196771 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:33.196771 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:33.196771 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:33.196771 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:33.196771 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:33.196771 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:33.196771 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:33.196771 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:33.196771 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:33.196771 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:33.196771 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:33.196771 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:48:33.196771 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:48:33.196771 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:48:33.197725 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:48:33.197725 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:33.197725 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:33.197725 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:33.197725 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:33.197725 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:33.197725 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:33.197725 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:33.197725 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:33.197725 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:33.197725 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:33.197725 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:33.197725 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:33.197725 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:33.197725 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:33.197725 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:33.197725 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:33.199208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:48:33.199208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:48:33.199208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:48:33.199208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:48:33.199208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:48:33.199208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:48:33.199208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:33.199208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:33.199208 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 290/589] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:48:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:33.199208 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07592070-4801-4c9b-84c7-01f72a5dbab4 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:33.232106 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4125bb2-6eab-4223-a0ea-cd930ff304b5 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:33.270279 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4aefa-ae73-4891-b5f9-757131d7da1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:33.270859 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4aefa-ae73-4891-b5f9-757131d7da1d None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:33.271141 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4aefa-ae73-4891-b5f9-757131d7da1d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:33.271420 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4aefa-ae73-4891-b5f9-757131d7da1d None admin] PATH_INFO: `/v3/projects/5100a8d316d243fb8f82e9044e1b3b06/users/1179503bdab547c99175a27b7ce27b9d/roles/6e3e398ba29e436e8532a45d2fdf1321` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:33.286634 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe4aefa-ae73-4891-b5f9-757131d7da1d None admin] RBAC: Authorizing `identity:create_grant(project_id=5100a8d316d243fb8f82e9044e1b3b06, user_id=1179503bdab547c99175a27b7ce27b9d, role_id=6e3e398ba29e436e8532a45d2fdf1321)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:33.288224 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe4aefa-ae73-4891-b5f9-757131d7da1d None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:33.309204 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07592070-4801-4c9b-84c7-01f72a5dbab4 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:33.343050 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f1f5b-a5fc-4026-8d33-e3e767af4fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:33.343664 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f1f5b-a5fc-4026-8d33-e3e767af4fab None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:33.344047 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f1f5b-a5fc-4026-8d33-e3e767af4fab None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:33.344323 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f1f5b-a5fc-4026-8d33-e3e767af4fab None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:33.351737 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f1f5b-a5fc-4026-8d33-e3e767af4fab None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:33.353181 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f1f5b-a5fc-4026-8d33-e3e767af4fab None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:33.367132 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 300/590] 10.222.0.27 () {58 vars in 1439 bytes} [Sun Sep 4 00:48:33 2022] PUT /identity/v3/projects/5100a8d316d243fb8f82e9044e1b3b06/users/1179503bdab547c99175a27b7ce27b9d/roles/6e3e398ba29e436e8532a45d2fdf1321 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 00:48:33.367132 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4aefa-ae73-4891-b5f9-757131d7da1d None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:33.367442 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-7d6f1f5b-a5fc-4026-8d33-e3e767af4fab None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 04 00:48:33.367442 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:33.367442 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:33.367442 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:33.367442 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:33.367442 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:33.367442 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:33.367442 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:33.367442 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:33.367442 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:33.367442 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:33.367442 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:33.367442 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:48:33.367442 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:48:33.367442 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:48:33.367442 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:48:33.368517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:33.368517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:33.368517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:48:33.368517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:33.368517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:48:33.368517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:33.368517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:33.368517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:33.368517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:33.368517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:33.368517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:33.368517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:33.368517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:33.368517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:33.368517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:33.368517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:33.368517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:33.368517 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:33.369663 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:33.369663 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:33.369663 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:48:33.369663 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:48:33.369663 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:48:33.369663 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:48:33.369663 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:48:33.369663 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:48:33.369663 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 04 00:48:33.369663 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:33.369663 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 291/591] 10.222.0.27 () {56 vars in 1102 bytes} [Sun Sep 4 00:48:33 2022] GET /identity/v3/users/neutron => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:33.369663 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f1f5b-a5fc-4026-8d33-e3e767af4fab None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:33.444767 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:33.497930 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f1f5b-a5fc-4026-8d33-e3e767af4fab None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:33.520820 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:33.525768 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4aefa-ae73-4891-b5f9-757131d7da1d None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:33.542030 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e606db65-8315-4b7b-8300-d8459a73a653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:33.542504 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606db65-8315-4b7b-8300-d8459a73a653 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:33.542951 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606db65-8315-4b7b-8300-d8459a73a653 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:33.543234 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606db65-8315-4b7b-8300-d8459a73a653 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:33.543931 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e606db65-8315-4b7b-8300-d8459a73a653 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:33.546969 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e606db65-8315-4b7b-8300-d8459a73a653 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:33.546969 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e606db65-8315-4b7b-8300-d8459a73a653 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:33.561278 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 292/592] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:48:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:33.562203 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e606db65-8315-4b7b-8300-d8459a73a653 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:33.580364 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecaf763-238f-4628-93c3-179c71516d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:33.581055 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecaf763-238f-4628-93c3-179c71516d81 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:33.581358 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecaf763-238f-4628-93c3-179c71516d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:33.581655 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecaf763-238f-4628-93c3-179c71516d81 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:33.582320 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecaf763-238f-4628-93c3-179c71516d81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:33.582624 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecaf763-238f-4628-93c3-179c71516d81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:33.584121 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecaf763-238f-4628-93c3-179c71516d81 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:33.602865 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 301/593] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:33.602865 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecaf763-238f-4628-93c3-179c71516d81 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:33.705896 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e606db65-8315-4b7b-8300-d8459a73a653 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:33.722368 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:33.724492 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecaf763-238f-4628-93c3-179c71516d81 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:33.743846 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ace93-b3e7-49fe-b832-353d0a83066d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:33.744725 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ace93-b3e7-49fe-b832-353d0a83066d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:33.745194 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ace93-b3e7-49fe-b832-353d0a83066d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:33.745597 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ace93-b3e7-49fe-b832-353d0a83066d None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:33.746449 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1ace93-b3e7-49fe-b832-353d0a83066d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:33.746889 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1ace93-b3e7-49fe-b832-353d0a83066d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:33.748564 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1ace93-b3e7-49fe-b832-353d0a83066d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:33.767712 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 293/594] 10.222.0.27 () {56 vars in 1139 bytes} [Sun Sep 4 00:48:33 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:33.768799 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ace93-b3e7-49fe-b832-353d0a83066d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:33.771868 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e2c02-ca9a-42c1-9529-ad7cfed4ad81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:33.773670 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e2c02-ca9a-42c1-9529-ad7cfed4ad81 None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:33.774109 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e2c02-ca9a-42c1-9529-ad7cfed4ad81 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:33.774533 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e2c02-ca9a-42c1-9529-ad7cfed4ad81 None admin] PATH_INFO: `/v3/projects/5100a8d316d243fb8f82e9044e1b3b06/users/68bc00bda5194cd8a7928f8c4df4ccd1/roles/6e3e398ba29e436e8532a45d2fdf1321` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:33.795015 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e2c02-ca9a-42c1-9529-ad7cfed4ad81 None admin] RBAC: Authorizing `identity:create_grant(project_id=5100a8d316d243fb8f82e9044e1b3b06, user_id=68bc00bda5194cd8a7928f8c4df4ccd1, role_id=6e3e398ba29e436e8532a45d2fdf1321)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:33.797652 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e2c02-ca9a-42c1-9529-ad7cfed4ad81 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:33.875088 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 302/595] 10.222.0.27 () {58 vars in 1439 bytes} [Sun Sep 4 00:48:33 2022] PUT /identity/v3/projects/5100a8d316d243fb8f82e9044e1b3b06/users/68bc00bda5194cd8a7928f8c4df4ccd1/roles/6e3e398ba29e436e8532a45d2fdf1321 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 00:48:33.876718 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e2c02-ca9a-42c1-9529-ad7cfed4ad81 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:33.889457 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ace93-b3e7-49fe-b832-353d0a83066d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:33.891701 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:33.989440 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf97dfb-ccb8-4a05-bc60-de0f0a4b0404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:33.990135 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf97dfb-ccb8-4a05-bc60-de0f0a4b0404 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:33.990417 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf97dfb-ccb8-4a05-bc60-de0f0a4b0404 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:33.990689 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf97dfb-ccb8-4a05-bc60-de0f0a4b0404 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:34.005689 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf97dfb-ccb8-4a05-bc60-de0f0a4b0404 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:34.007206 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf97dfb-ccb8-4a05-bc60-de0f0a4b0404 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:34.015888 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e2c02-ca9a-42c1-9529-ad7cfed4ad81 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:34.034671 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-5bf97dfb-ccb8-4a05-bc60-de0f0a4b0404 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 04 00:48:34.034671 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:34.034671 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:34.034671 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:34.034671 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:34.034671 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:34.034671 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:34.034671 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:34.034671 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:34.034671 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:34.034671 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:34.034671 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:34.034671 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:48:34.034671 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:48:34.034671 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:48:34.034671 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:48:34.036300 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:34.036300 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:34.036300 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:48:34.036300 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:34.036300 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:48:34.036300 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:34.036300 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:34.036300 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:34.036300 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:34.036300 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:34.036300 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:34.036300 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:34.036300 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:34.036300 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:34.036300 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:34.036300 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:34.036300 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:34.036300 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:34.037491 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:34.037491 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:34.037491 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:48:34.037491 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:48:34.037491 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:48:34.037491 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:48:34.037491 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:48:34.037491 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:48:34.037491 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 04 00:48:34.037491 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:34.037491 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 294/596] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:48:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:34.041868 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf97dfb-ccb8-4a05-bc60-de0f0a4b0404 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:34.061902 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad430eee-ab98-4469-8044-477b65c1df55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:34.063069 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad430eee-ab98-4469-8044-477b65c1df55 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:34.063791 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad430eee-ab98-4469-8044-477b65c1df55 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:34.064261 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad430eee-ab98-4469-8044-477b65c1df55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:34.079159 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad430eee-ab98-4469-8044-477b65c1df55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:34.080809 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:34.082322 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad430eee-ab98-4469-8044-477b65c1df55 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:34.103327 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-ad430eee-ab98-4469-8044-477b65c1df55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:34.103327 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:34.103327 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:34.103327 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:34.103327 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:34.103327 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:34.103327 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:34.103327 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:34.103327 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:34.103327 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:34.103327 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:34.103327 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:34.103327 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:34.103327 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:34.103327 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:34.103327 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:34.104329 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:34.104329 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:34.104329 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:34.104329 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:34.104329 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:34.104329 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:34.104329 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:34.104329 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:34.104329 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:34.104329 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:34.104329 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:34.104329 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:34.104329 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:34.104329 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:34.104329 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:34.104329 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:34.104329 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:34.105237 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:34.105237 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:34.105237 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:34.106173 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 303/597] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:34.112370 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad430eee-ab98-4469-8044-477b65c1df55 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:34.198635 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf97dfb-ccb8-4a05-bc60-de0f0a4b0404 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:34.214308 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:34.268901 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1d70a-1747-42fa-909b-c0b07b474a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:34.269512 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1d70a-1747-42fa-909b-c0b07b474a53 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:34.269791 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1d70a-1747-42fa-909b-c0b07b474a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:34.270059 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1d70a-1747-42fa-909b-c0b07b474a53 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:34.270968 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1d70a-1747-42fa-909b-c0b07b474a53 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:34.271279 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1d70a-1747-42fa-909b-c0b07b474a53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:34.272727 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1d70a-1747-42fa-909b-c0b07b474a53 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:34.291935 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad430eee-ab98-4469-8044-477b65c1df55 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:34.308856 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 295/598] 10.222.0.27 () {56 vars in 1137 bytes} [Sun Sep 4 00:48:34 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:48:34.315737 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1d70a-1747-42fa-909b-c0b07b474a53 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:34.362028 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e96a8a-446a-40cc-9947-55c2a817d3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:34.363487 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e96a8a-446a-40cc-9947-55c2a817d3b0 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:34.363772 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e96a8a-446a-40cc-9947-55c2a817d3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:34.364052 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e96a8a-446a-40cc-9947-55c2a817d3b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:34.364814 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e96a8a-446a-40cc-9947-55c2a817d3b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:34.365792 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e96a8a-446a-40cc-9947-55c2a817d3b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:34.367398 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e96a8a-446a-40cc-9947-55c2a817d3b0 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:34.382848 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 304/599] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:48:34.387815 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e96a8a-446a-40cc-9947-55c2a817d3b0 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:34.504563 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:34.515003 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1d70a-1747-42fa-909b-c0b07b474a53 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:34.543682 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e96a8a-446a-40cc-9947-55c2a817d3b0 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:34.550594 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc93a0-914b-4954-844b-6c0e5f0bf161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:34.551364 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc93a0-914b-4954-844b-6c0e5f0bf161 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:34.551748 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc93a0-914b-4954-844b-6c0e5f0bf161 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:34.552124 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc93a0-914b-4954-844b-6c0e5f0bf161 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:34.562915 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fc93a0-914b-4954-844b-6c0e5f0bf161 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:34.564567 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fc93a0-914b-4954-844b-6c0e5f0bf161 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:34.567584 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:34.583966 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8707bf29-d816-4698-b62b-e409dad501c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:34.588698 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707bf29-d816-4698-b62b-e409dad501c9 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:34.588698 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707bf29-d816-4698-b62b-e409dad501c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:34.588698 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707bf29-d816-4698-b62b-e409dad501c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:34.593707 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8707bf29-d816-4698-b62b-e409dad501c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:34.594796 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-a2fc93a0-914b-4954-844b-6c0e5f0bf161 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:34.594796 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:34.594796 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:34.594796 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:34.594796 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:34.594796 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:34.594796 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:34.594796 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:34.594796 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:34.594796 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:34.594796 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:34.594796 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:34.594796 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:34.594796 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:34.594796 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:34.594796 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:34.596274 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:34.596274 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:34.596274 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:34.596274 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:34.596274 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:34.596274 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:34.596274 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:34.596274 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:34.596274 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:34.596274 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:34.596274 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:34.596274 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:34.596274 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:34.596274 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:34.596274 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:34.596274 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:34.596274 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:34.597364 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:34.597364 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:34.597364 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:34.597364 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 296/600] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:34.598502 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8707bf29-d816-4698-b62b-e409dad501c9 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:34.602797 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc93a0-914b-4954-844b-6c0e5f0bf161 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:34.611830 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-8707bf29-d816-4698-b62b-e409dad501c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:34.611830 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:34.611830 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:34.611830 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:34.611830 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:34.611830 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:34.611830 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:34.611830 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:34.611830 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:34.611830 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:34.611830 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:34.611830 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:34.611830 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:48:34.611830 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:48:34.611830 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:48:34.612734 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:48:34.612734 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:34.612734 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:34.612734 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:34.612734 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:34.612734 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:34.612734 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:34.612734 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:34.612734 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:34.612734 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:34.612734 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:34.612734 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:34.612734 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:34.612734 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:34.612734 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:34.612734 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:34.612734 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:34.613605 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:48:34.613605 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:48:34.613605 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:48:34.613605 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:48:34.613605 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:48:34.613605 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:48:34.613605 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:34.613605 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:34.615740 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 305/601] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:48:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:34.621083 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f9998-a86a-4a2a-aaff-5084adc4f161 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:34.621365 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f9998-a86a-4a2a-aaff-5084adc4f161 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:34.621639 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f9998-a86a-4a2a-aaff-5084adc4f161 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:34.623006 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 306/602] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 00:48:34.628667 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931de769-375e-4434-81c2-3ae2981b6329 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:34.629201 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931de769-375e-4434-81c2-3ae2981b6329 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:34.629691 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931de769-375e-4434-81c2-3ae2981b6329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:34.655060 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-931de769-375e-4434-81c2-3ae2981b6329 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:34.657251 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-931de769-375e-4434-81c2-3ae2981b6329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:34.752618 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 307/603] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 04 00:48:34.752813 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc93a0-914b-4954-844b-6c0e5f0bf161 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:34.752932 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931de769-375e-4434-81c2-3ae2981b6329 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:34.815642 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7055709-aa4c-4bc1-a956-5ce772a730d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:34.816536 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7055709-aa4c-4bc1-a956-5ce772a730d2 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:34.816960 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7055709-aa4c-4bc1-a956-5ce772a730d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:34.817370 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7055709-aa4c-4bc1-a956-5ce772a730d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:34.818164 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7055709-aa4c-4bc1-a956-5ce772a730d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:34.818608 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7055709-aa4c-4bc1-a956-5ce772a730d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:34.820103 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7055709-aa4c-4bc1-a956-5ce772a730d2 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:34.840381 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 297/604] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:34.853476 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7055709-aa4c-4bc1-a956-5ce772a730d2 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:34.856507 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:34.904760 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931de769-375e-4434-81c2-3ae2981b6329 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:34.918969 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:34.947364 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9afcc3-c9d7-4bc9-9e17-3d8383d233b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:34.948259 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9afcc3-c9d7-4bc9-9e17-3d8383d233b9 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:34.948615 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9afcc3-c9d7-4bc9-9e17-3d8383d233b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:34.948890 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9afcc3-c9d7-4bc9-9e17-3d8383d233b9 None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:34.949589 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9afcc3-c9d7-4bc9-9e17-3d8383d233b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:34.949896 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9afcc3-c9d7-4bc9-9e17-3d8383d233b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:34.952626 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9afcc3-c9d7-4bc9-9e17-3d8383d233b9 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:34.968430 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 308/605] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:48:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:34.979349 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9afcc3-c9d7-4bc9-9e17-3d8383d233b9 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:35.035589 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7055709-aa4c-4bc1-a956-5ce772a730d2 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:35.076067 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe8d77-a5cd-4520-9476-7462781569d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:35.076067 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe8d77-a5cd-4520-9476-7462781569d8 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:35.076357 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe8d77-a5cd-4520-9476-7462781569d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:35.076444 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe8d77-a5cd-4520-9476-7462781569d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:35.084177 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fe8d77-a5cd-4520-9476-7462781569d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:35.085727 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fe8d77-a5cd-4520-9476-7462781569d8 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:35.098838 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-d8fe8d77-a5cd-4520-9476-7462781569d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:35.098838 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:35.098838 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:35.098838 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:35.098838 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:35.098838 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:35.098838 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:35.098838 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:35.098838 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:35.098838 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:35.098838 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:35.098838 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:35.098838 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:48:35.098838 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:48:35.098838 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:48:35.099773 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:48:35.099773 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:35.099773 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:35.099773 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:35.099773 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:35.099773 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:35.099773 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:35.099773 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:35.099773 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:35.099773 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:35.099773 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:35.099773 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:35.099773 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:35.099773 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:35.099773 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:35.099773 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:35.099773 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:35.101047 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:48:35.101047 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:48:35.101047 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:48:35.101047 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:48:35.101047 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:48:35.101047 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:48:35.101047 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:35.101047 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:35.101047 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 298/606] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:48:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:35.106077 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe8d77-a5cd-4520-9476-7462781569d8 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:35.173075 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:35.214509 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9afcc3-c9d7-4bc9-9e17-3d8383d233b9 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:35.217768 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe8d77-a5cd-4520-9476-7462781569d8 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:35.257416 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41380d9-1d9d-4307-97e5-38347ba41ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:35.258026 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41380d9-1d9d-4307-97e5-38347ba41ec5 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:35.258311 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41380d9-1d9d-4307-97e5-38347ba41ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:35.258590 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41380d9-1d9d-4307-97e5-38347ba41ec5 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:35.259300 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41380d9-1d9d-4307-97e5-38347ba41ec5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:35.259589 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41380d9-1d9d-4307-97e5-38347ba41ec5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:35.261053 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41380d9-1d9d-4307-97e5-38347ba41ec5 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:35.266060 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ec1d7-52eb-482f-8781-eae104354eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:35.266945 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ec1d7-52eb-482f-8781-eae104354eb8 None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:35.267997 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ec1d7-52eb-482f-8781-eae104354eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:35.268563 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ec1d7-52eb-482f-8781-eae104354eb8 None admin] PATH_INFO: `/v3/projects/5100a8d316d243fb8f82e9044e1b3b06/users/861c278f8bf34b3bb407b2e051119ee9/roles/6e3e398ba29e436e8532a45d2fdf1321` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:35.274504 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 299/607] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:48:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:35.275334 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41380d9-1d9d-4307-97e5-38347ba41ec5 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:35.287594 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5ec1d7-52eb-482f-8781-eae104354eb8 None admin] RBAC: Authorizing `identity:create_grant(project_id=5100a8d316d243fb8f82e9044e1b3b06, user_id=861c278f8bf34b3bb407b2e051119ee9, role_id=6e3e398ba29e436e8532a45d2fdf1321)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:35.290353 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5ec1d7-52eb-482f-8781-eae104354eb8 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:35.386232 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 309/608] 10.222.0.27 () {58 vars in 1439 bytes} [Sun Sep 4 00:48:34 2022] PUT /identity/v3/projects/5100a8d316d243fb8f82e9044e1b3b06/users/861c278f8bf34b3bb407b2e051119ee9/roles/6e3e398ba29e436e8532a45d2fdf1321 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 00:48:35.386421 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41380d9-1d9d-4307-97e5-38347ba41ec5 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:35.387512 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ec1d7-52eb-482f-8781-eae104354eb8 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:35.416990 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:35.425221 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:35.431560 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaaa4b6-cbfb-4636-9de3-1f4197f82b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:35.432422 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaaa4b6-cbfb-4636-9de3-1f4197f82b83 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:35.432787 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaaa4b6-cbfb-4636-9de3-1f4197f82b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:35.433183 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaaa4b6-cbfb-4636-9de3-1f4197f82b83 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:35.447131 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaaa4b6-cbfb-4636-9de3-1f4197f82b83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:35.452878 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-aaaaa4b6-cbfb-4636-9de3-1f4197f82b83 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:35.452878 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:35.452878 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:35.452878 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:35.452878 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:35.452878 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:35.452878 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:35.452878 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:35.452878 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:35.452878 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:35.452878 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:35.452878 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:35.452878 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:35.452878 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:35.452878 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:35.452878 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:35.454026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:35.454026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:35.454026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:35.454026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:35.454026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:35.454026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:35.454026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:35.454026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:35.454026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:35.454026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:35.454026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:35.454026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:35.454026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:35.454026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:35.454026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:35.454026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:35.454889 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:35.454889 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:35.454889 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:35.454889 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:35.454889 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:35.454889 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:35.454889 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:35.454889 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:35.454889 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:35.454889 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:35.454889 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:35.454889 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:35.454889 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 300/609] 10.222.0.27 () {56 vars in 1102 bytes} [Sun Sep 4 00:48:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:35.461861 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973ca3d-8e08-41ea-a832-3d27ea074519 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:35.462834 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973ca3d-8e08-41ea-a832-3d27ea074519 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:35.463532 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973ca3d-8e08-41ea-a832-3d27ea074519 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:35.465688 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 301/610] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:35 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:35.466646 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a973ca3d-8e08-41ea-a832-3d27ea074519 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:35.548453 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ec1d7-52eb-482f-8781-eae104354eb8 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:35.598787 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e1ba1-cda5-4351-a74b-3e88da821668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:35.599398 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e1ba1-cda5-4351-a74b-3e88da821668 None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:35.599678 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e1ba1-cda5-4351-a74b-3e88da821668 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:35.599949 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e1ba1-cda5-4351-a74b-3e88da821668 None admin] PATH_INFO: `/v3/projects/5100a8d316d243fb8f82e9044e1b3b06/users/6297dbc521e94330ae9294b942144181/roles/6e3e398ba29e436e8532a45d2fdf1321` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:35.600310 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a973ca3d-8e08-41ea-a832-3d27ea074519 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:35.616968 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4e1ba1-cda5-4351-a74b-3e88da821668 None admin] RBAC: Authorizing `identity:create_grant(project_id=5100a8d316d243fb8f82e9044e1b3b06, user_id=6297dbc521e94330ae9294b942144181, role_id=6e3e398ba29e436e8532a45d2fdf1321)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:35.618554 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4e1ba1-cda5-4351-a74b-3e88da821668 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:35.640657 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b1a2c-4c7a-416e-8ee0-42ea811d68bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:35.641167 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b1a2c-4c7a-416e-8ee0-42ea811d68bc None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:35.641443 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b1a2c-4c7a-416e-8ee0-42ea811d68bc None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:35.641734 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b1a2c-4c7a-416e-8ee0-42ea811d68bc None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:35.642388 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b1a2c-4c7a-416e-8ee0-42ea811d68bc None admin] RBAC: Adding query filter params (name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:35.642765 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b1a2c-4c7a-416e-8ee0-42ea811d68bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:35.644121 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b1a2c-4c7a-416e-8ee0-42ea811d68bc None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:35.657166 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 302/611] 10.222.0.27 () {56 vars in 1103 bytes} [Sun Sep 4 00:48:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:35.657166 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fae3c-9556-4b1d-b28c-2dc061d5b17b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:35.657166 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fae3c-9556-4b1d-b28c-2dc061d5b17b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:35.657166 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fae3c-9556-4b1d-b28c-2dc061d5b17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:35.675583 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-918fae3c-9556-4b1d-b28c-2dc061d5b17b None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:35.676857 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-918fae3c-9556-4b1d-b28c-2dc061d5b17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:35.691645 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 310/612] 10.222.0.27 () {58 vars in 1439 bytes} [Sun Sep 4 00:48:35 2022] PUT /identity/v3/projects/5100a8d316d243fb8f82e9044e1b3b06/users/6297dbc521e94330ae9294b942144181/roles/6e3e398ba29e436e8532a45d2fdf1321 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 00:48:35.693042 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e1ba1-cda5-4351-a74b-3e88da821668 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:35.732428 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:35.768993 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 303/613] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:35.831148 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:35.869269 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e1ba1-cda5-4351-a74b-3e88da821668 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:35.919083 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86662c-89ff-4513-a71f-03292ef2286c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:35.919645 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86662c-89ff-4513-a71f-03292ef2286c None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:35.919950 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86662c-89ff-4513-a71f-03292ef2286c None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:35.920229 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86662c-89ff-4513-a71f-03292ef2286c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:35.935353 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c86662c-89ff-4513-a71f-03292ef2286c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:35.936876 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c86662c-89ff-4513-a71f-03292ef2286c None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:35.958853 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-4c86662c-89ff-4513-a71f-03292ef2286c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:35.958853 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:35.958853 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:35.958853 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:35.958853 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:35.958853 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:35.958853 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:35.958853 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:35.958853 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:35.958853 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:35.958853 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:35.958853 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:35.958853 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:35.958853 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:35.958853 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:35.958853 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:35.959865 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:35.959865 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:35.959865 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:35.959865 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:35.959865 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:35.959865 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:35.959865 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:35.959865 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:35.959865 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:35.959865 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:35.959865 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:35.959865 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:35.959865 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:35.959865 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:35.959865 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:35.959865 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:35.959865 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:35.960802 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:35.960802 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:35.960802 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:35.960802 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 311/614] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:35.967095 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918fae3c-9556-4b1d-b28c-2dc061d5b17b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:36.066459 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86662c-89ff-4513-a71f-03292ef2286c None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:36.088737 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918fae3c-9556-4b1d-b28c-2dc061d5b17b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:36.129225 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1036f0-a2fd-4f43-b4c6-2b363de455d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:36.130002 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1036f0-a2fd-4f43-b4c6-2b363de455d6 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:36.130380 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1036f0-a2fd-4f43-b4c6-2b363de455d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:36.130754 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1036f0-a2fd-4f43-b4c6-2b363de455d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:36.131604 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1036f0-a2fd-4f43-b4c6-2b363de455d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:36.132010 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1036f0-a2fd-4f43-b4c6-2b363de455d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:36.133671 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1036f0-a2fd-4f43-b4c6-2b363de455d6 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:36.147796 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 304/615] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:36.152653 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a07070-6de5-4f45-abef-79aa79fabadf None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:36.152653 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a07070-6de5-4f45-abef-79aa79fabadf None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:36.152653 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a07070-6de5-4f45-abef-79aa79fabadf None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:36.152653 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 305/616] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:36.155496 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a07070-6de5-4f45-abef-79aa79fabadf None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:36.184199 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86662c-89ff-4513-a71f-03292ef2286c None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:36.195020 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:36.209744 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:36.222443 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766618ce-cc5e-45f7-81d4-1846b090274f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:36.223925 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766618ce-cc5e-45f7-81d4-1846b090274f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:36.224489 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766618ce-cc5e-45f7-81d4-1846b090274f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:36.224971 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766618ce-cc5e-45f7-81d4-1846b090274f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:36.238425 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766618ce-cc5e-45f7-81d4-1846b090274f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:36.243795 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-766618ce-cc5e-45f7-81d4-1846b090274f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:36.243795 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:36.243795 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:36.243795 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:36.243795 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:36.243795 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:36.243795 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:36.243795 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:36.243795 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:36.243795 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:36.243795 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:36.243795 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:36.243795 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:36.243795 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:36.243795 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:36.243795 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:36.251202 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:36.251202 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:36.251202 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:36.251202 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:36.251202 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:36.251202 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:36.251202 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:36.251202 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:36.251202 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:36.251202 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:36.251202 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:36.251202 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:36.251202 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:36.251202 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:36.251202 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:36.251202 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:36.252630 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:36.252630 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:36.252630 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:36.252630 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:36.252630 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:36.252630 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:36.252630 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:36.252630 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:36.252630 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:36.252630 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:36.252630 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:36.252630 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:36.252630 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 312/617] 10.222.0.27 () {56 vars in 1102 bytes} [Sun Sep 4 00:48:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:48:36.255100 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e17d9aa-c903-4348-b9fd-80ffb0ededb8 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:36.255605 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e17d9aa-c903-4348-b9fd-80ffb0ededb8 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:36.256083 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e17d9aa-c903-4348-b9fd-80ffb0ededb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:36.272451 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a07070-6de5-4f45-abef-79aa79fabadf None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:36.276034 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-4e17d9aa-c903-4348-b9fd-80ffb0ededb8 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:36.277497 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-4e17d9aa-c903-4348-b9fd-80ffb0ededb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:36.309693 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a914a-f0be-4cc9-be7e-3b885394159a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:36.310323 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a914a-f0be-4cc9-be7e-3b885394159a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:36.310605 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a914a-f0be-4cc9-be7e-3b885394159a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:36.310938 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a914a-f0be-4cc9-be7e-3b885394159a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:36.319122 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9a914a-f0be-4cc9-be7e-3b885394159a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:36.320707 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9a914a-f0be-4cc9-be7e-3b885394159a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:36.334232 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-ff9a914a-f0be-4cc9-be7e-3b885394159a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 04 00:48:36.334232 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:36.334232 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:36.334232 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:36.334232 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:36.334232 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:36.334232 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:36.334232 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:36.334232 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:36.334232 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:36.334232 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:36.334232 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:36.334232 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:48:36.334232 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:48:36.334232 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:48:36.334232 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:48:36.335208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:36.335208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:36.335208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:48:36.335208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:36.335208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:48:36.335208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:36.335208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:36.335208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:36.335208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:36.335208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:36.335208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:36.335208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:36.335208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:36.335208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:36.335208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:36.335208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:36.335208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:36.335208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:36.336153 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:36.336153 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:36.336153 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:48:36.336153 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:48:36.336153 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:48:36.336153 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:48:36.336153 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:48:36.336153 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:48:36.336153 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 04 00:48:36.336153 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:36.340179 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 306/618] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:48:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:36.341118 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a914a-f0be-4cc9-be7e-3b885394159a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:36.354997 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 313/619] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:36.356137 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e17d9aa-c903-4348-b9fd-80ffb0ededb8 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:36.488252 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e17d9aa-c903-4348-b9fd-80ffb0ededb8 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:36.502420 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:36.524154 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:36.534007 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a914a-f0be-4cc9-be7e-3b885394159a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:36.536034 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572357d4-20e0-44a2-8cd7-6e4c6420a410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:36.536679 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572357d4-20e0-44a2-8cd7-6e4c6420a410 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:36.536982 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572357d4-20e0-44a2-8cd7-6e4c6420a410 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:36.537259 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572357d4-20e0-44a2-8cd7-6e4c6420a410 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:36.538003 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572357d4-20e0-44a2-8cd7-6e4c6420a410 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:36.543476 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572357d4-20e0-44a2-8cd7-6e4c6420a410 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:36.543476 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572357d4-20e0-44a2-8cd7-6e4c6420a410 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:36.577654 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 314/620] 10.222.0.27 () {56 vars in 1135 bytes} [Sun Sep 4 00:48:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:36.583955 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572357d4-20e0-44a2-8cd7-6e4c6420a410 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:36.602625 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900d552-eda6-4246-ad2f-240e48fc1da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:36.603240 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900d552-eda6-4246-ad2f-240e48fc1da7 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:36.603522 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900d552-eda6-4246-ad2f-240e48fc1da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:36.603798 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900d552-eda6-4246-ad2f-240e48fc1da7 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:36.604489 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3900d552-eda6-4246-ad2f-240e48fc1da7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:36.604797 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3900d552-eda6-4246-ad2f-240e48fc1da7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:36.606164 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3900d552-eda6-4246-ad2f-240e48fc1da7 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:36.622665 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 307/621] 10.222.0.27 () {56 vars in 1103 bytes} [Sun Sep 4 00:48:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:36.628886 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900d552-eda6-4246-ad2f-240e48fc1da7 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:36.740415 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572357d4-20e0-44a2-8cd7-6e4c6420a410 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:36.766377 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900d552-eda6-4246-ad2f-240e48fc1da7 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:36.778446 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd16de2-9ad2-4a82-95a7-a599be01fbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:36.779075 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd16de2-9ad2-4a82-95a7-a599be01fbb7 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:36.779358 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd16de2-9ad2-4a82-95a7-a599be01fbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:36.779635 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd16de2-9ad2-4a82-95a7-a599be01fbb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:36.791603 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd16de2-9ad2-4a82-95a7-a599be01fbb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:36.793201 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd16de2-9ad2-4a82-95a7-a599be01fbb7 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:36.802883 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a8a160-ce2f-4a0e-9ade-d6d04bdfad01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:36.803559 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a8a160-ce2f-4a0e-9ade-d6d04bdfad01 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:36.803837 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a8a160-ce2f-4a0e-9ade-d6d04bdfad01 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:36.804109 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a8a160-ce2f-4a0e-9ade-d6d04bdfad01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:36.808958 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-6dd16de2-9ad2-4a82-95a7-a599be01fbb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:36.808958 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:36.808958 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:36.808958 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:36.808958 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:36.808958 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:36.808958 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:36.808958 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:36.808958 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:36.808958 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:36.808958 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:36.808958 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:36.808958 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:36.808958 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:36.808958 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:36.808958 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:36.809903 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:36.809903 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:36.809903 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:36.809903 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:36.809903 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:36.809903 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:36.809903 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:36.809903 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:36.809903 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:36.809903 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:36.809903 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:36.809903 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:36.809903 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:36.809903 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:36.809903 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:36.809903 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:36.809903 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:36.810788 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:36.810788 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:36.810788 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:36.810788 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 315/622] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:48:36.811475 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd16de2-9ad2-4a82-95a7-a599be01fbb7 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:36.815659 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a8a160-ce2f-4a0e-9ade-d6d04bdfad01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:36.817293 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a8a160-ce2f-4a0e-9ade-d6d04bdfad01 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:36.833486 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-23a8a160-ce2f-4a0e-9ade-d6d04bdfad01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:36.833486 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:36.833486 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:36.833486 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:36.833486 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:36.833486 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:36.833486 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:36.833486 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:36.833486 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:36.833486 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:36.833486 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:36.833486 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:36.833486 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:36.833486 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:36.833486 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:36.833486 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:36.834455 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:36.834455 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:36.834455 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:36.834455 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:36.834455 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:36.834455 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:36.834455 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:36.834455 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:36.834455 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:36.834455 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:36.834455 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:36.834455 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:36.834455 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:36.834455 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:36.834455 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:36.834455 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:36.834455 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:36.835312 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:36.835312 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:36.835312 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:36.835312 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 308/623] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:36.836015 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a8a160-ce2f-4a0e-9ade-d6d04bdfad01 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:36.837291 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:36.849756 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:36.921169 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd16de2-9ad2-4a82-95a7-a599be01fbb7 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:36.946073 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a8a160-ce2f-4a0e-9ade-d6d04bdfad01 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:36.960017 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d24d3d-0192-4ab2-a7be-c19362b593bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:36.960685 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d24d3d-0192-4ab2-a7be-c19362b593bd None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:36.961074 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d24d3d-0192-4ab2-a7be-c19362b593bd None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:36.961396 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d24d3d-0192-4ab2-a7be-c19362b593bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:36.972974 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d24d3d-0192-4ab2-a7be-c19362b593bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:36.978246 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-76d24d3d-0192-4ab2-a7be-c19362b593bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:36.978246 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:36.978246 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:36.978246 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:36.978246 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:36.978246 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:36.978246 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:36.978246 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:36.978246 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:36.978246 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:36.978246 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:36.978246 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:36.978246 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:36.978246 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:36.978246 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:36.978246 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:36.979380 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:36.979380 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:36.979380 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:36.979380 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:36.979380 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:36.979380 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:36.979380 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:36.979380 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:36.979380 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:36.979380 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:36.979380 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:36.979380 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:36.979380 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:36.979380 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:36.979380 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:36.979380 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:36.980370 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:36.980370 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:36.980370 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:36.980370 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:36.980370 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:36.980370 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:36.980370 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:36.980370 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:36.980370 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:36.980370 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:36.980370 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:36.980370 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:36.980370 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 316/624] 10.222.0.27 () {56 vars in 1102 bytes} [Sun Sep 4 00:48:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:36.981272 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d24d3d-0192-4ab2-a7be-c19362b593bd None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:36.988293 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf322b0-c676-4d23-ab59-4c5f583b1863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:36.989250 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf322b0-c676-4d23-ab59-4c5f583b1863 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:36.989732 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf322b0-c676-4d23-ab59-4c5f583b1863 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:36.990205 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf322b0-c676-4d23-ab59-4c5f583b1863 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:36.991213 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf322b0-c676-4d23-ab59-4c5f583b1863 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:36.991741 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf322b0-c676-4d23-ab59-4c5f583b1863 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:36.993673 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf322b0-c676-4d23-ab59-4c5f583b1863 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:37.008206 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 309/625] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:37.009552 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf322b0-c676-4d23-ab59-4c5f583b1863 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:37.071229 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:37.085724 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:37.091257 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d24d3d-0192-4ab2-a7be-c19362b593bd None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:37.131579 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba42307-3b81-42ca-a7bd-bac045b6344a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:37.132202 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba42307-3b81-42ca-a7bd-bac045b6344a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:37.132525 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba42307-3b81-42ca-a7bd-bac045b6344a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:37.132868 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba42307-3b81-42ca-a7bd-bac045b6344a None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:37.133497 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba42307-3b81-42ca-a7bd-bac045b6344a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:37.133804 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba42307-3b81-42ca-a7bd-bac045b6344a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:37.135260 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba42307-3b81-42ca-a7bd-bac045b6344a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:37.135535 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf322b0-c676-4d23-ab59-4c5f583b1863 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:37.149152 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 317/626] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:37.150028 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba42307-3b81-42ca-a7bd-bac045b6344a None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:37.186568 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec4b30-94bd-47fd-94f4-487b8a60c0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:37.188134 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec4b30-94bd-47fd-94f4-487b8a60c0a5 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:37.188773 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec4b30-94bd-47fd-94f4-487b8a60c0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:37.189458 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec4b30-94bd-47fd-94f4-487b8a60c0a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:37.190862 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec4b30-94bd-47fd-94f4-487b8a60c0a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:37.191754 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec4b30-94bd-47fd-94f4-487b8a60c0a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:37.193809 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec4b30-94bd-47fd-94f4-487b8a60c0a5 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:37.206421 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 310/627] 10.222.0.27 () {56 vars in 1103 bytes} [Sun Sep 4 00:48:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:37.207849 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec4b30-94bd-47fd-94f4-487b8a60c0a5 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:37.275890 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba42307-3b81-42ca-a7bd-bac045b6344a None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:37.313092 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e2fe2b-40c5-4d6b-82e3-358738d282d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:37.314050 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2fe2b-40c5-4d6b-82e3-358738d282d3 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:37.314556 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2fe2b-40c5-4d6b-82e3-358738d282d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:37.315060 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2fe2b-40c5-4d6b-82e3-358738d282d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:37.322785 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e2fe2b-40c5-4d6b-82e3-358738d282d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:37.324707 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e2fe2b-40c5-4d6b-82e3-358738d282d3 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:37.337799 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-47e2fe2b-40c5-4d6b-82e3-358738d282d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:37.337799 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:37.337799 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:37.337799 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:37.337799 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:37.337799 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:37.337799 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:37.337799 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:37.337799 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:37.337799 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:37.337799 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:37.337799 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:37.337799 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:48:37.337799 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:48:37.337799 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:48:37.338784 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:48:37.338784 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:37.338784 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:37.338784 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:37.338784 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:37.338784 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:37.338784 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:37.338784 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:37.338784 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:37.338784 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:37.338784 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:37.338784 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:37.338784 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:37.338784 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:37.338784 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:37.338784 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:37.338784 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:37.339693 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:48:37.339693 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:48:37.339693 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:48:37.339693 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:48:37.339693 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:48:37.339693 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:48:37.339693 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:37.339693 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:37.340817 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec4b30-94bd-47fd-94f4-487b8a60c0a5 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:37.342374 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 318/628] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:48:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:37.343610 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e2fe2b-40c5-4d6b-82e3-358738d282d3 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:37.347480 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:37.380686 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:37.390762 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a04aec7-4366-436b-93c6-2c2573427d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:37.391400 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04aec7-4366-436b-93c6-2c2573427d10 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:37.391681 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04aec7-4366-436b-93c6-2c2573427d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:37.391958 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04aec7-4366-436b-93c6-2c2573427d10 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:37.400436 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a04aec7-4366-436b-93c6-2c2573427d10 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:37.401920 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a04aec7-4366-436b-93c6-2c2573427d10 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:37.418304 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-2a04aec7-4366-436b-93c6-2c2573427d10 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 04 00:48:37.418304 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:37.418304 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:37.418304 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:37.418304 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:37.418304 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:37.418304 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:37.418304 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:37.418304 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:37.418304 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:37.418304 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:37.418304 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:37.418304 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:48:37.418304 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:48:37.418304 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:48:37.418304 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:48:37.419320 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:37.419320 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:37.419320 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:48:37.419320 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:37.419320 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:48:37.419320 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:37.419320 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:37.419320 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:37.419320 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:37.419320 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:37.419320 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:37.419320 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:37.419320 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:37.419320 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:37.419320 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:37.419320 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:37.419320 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:37.419320 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:37.420267 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:37.420267 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:37.420267 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:48:37.420267 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:48:37.420267 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:48:37.420267 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:48:37.420267 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:48:37.420267 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:48:37.420267 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 04 00:48:37.420267 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:37.420267 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 311/629] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:48:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:37.420905 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a04aec7-4366-436b-93c6-2c2573427d10 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:37.484476 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e2fe2b-40c5-4d6b-82e3-358738d282d3 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:37.520756 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e43ad3-9606-457c-8ae8-08976df19663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:37.521640 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e43ad3-9606-457c-8ae8-08976df19663 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:37.521924 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e43ad3-9606-457c-8ae8-08976df19663 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:37.522205 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e43ad3-9606-457c-8ae8-08976df19663 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:37.532975 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e43ad3-9606-457c-8ae8-08976df19663 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:37.535146 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e43ad3-9606-457c-8ae8-08976df19663 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:37.541345 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a04aec7-4366-436b-93c6-2c2573427d10 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:37.550598 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-62e43ad3-9606-457c-8ae8-08976df19663 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:37.550598 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:37.550598 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:37.550598 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:37.550598 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:37.550598 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:37.550598 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:37.550598 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:37.550598 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:37.550598 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:37.550598 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:37.550598 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:37.550598 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:37.550598 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:37.550598 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:37.550598 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:37.551525 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:37.551525 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:37.551525 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:37.551525 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:37.551525 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:37.551525 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:37.551525 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:37.551525 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:37.551525 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:37.551525 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:37.551525 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:37.551525 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:37.551525 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:37.551525 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:37.551525 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:37.551525 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:37.551525 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:37.552417 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:37.552417 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:37.552417 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:37.552417 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 319/630] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:37.552684 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e43ad3-9606-457c-8ae8-08976df19663 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:37.584263 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857d2b4-fc6d-4953-9f12-b5e5a3b4e5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:37.584985 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857d2b4-fc6d-4953-9f12-b5e5a3b4e5fc None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:37.585266 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857d2b4-fc6d-4953-9f12-b5e5a3b4e5fc None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:37.585541 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857d2b4-fc6d-4953-9f12-b5e5a3b4e5fc None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:37.586233 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b857d2b4-fc6d-4953-9f12-b5e5a3b4e5fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:37.586536 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b857d2b4-fc6d-4953-9f12-b5e5a3b4e5fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:37.588080 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b857d2b4-fc6d-4953-9f12-b5e5a3b4e5fc None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:37.607729 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 312/631] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:48:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:37.608872 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af8b4b-9fd8-485c-bd04-f14acbfb9b2a None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:37.609162 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af8b4b-9fd8-485c-bd04-f14acbfb9b2a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:37.609459 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af8b4b-9fd8-485c-bd04-f14acbfb9b2a None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:37.610389 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:37.610683 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 313/632] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:37.611414 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86af8b4b-9fd8-485c-bd04-f14acbfb9b2a None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:37.664549 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e43ad3-9606-457c-8ae8-08976df19663 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:37.670223 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:37.699678 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aba4a8-4b91-4ea1-a5de-0654ae8a4e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:37.700288 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aba4a8-4b91-4ea1-a5de-0654ae8a4e92 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:37.700607 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aba4a8-4b91-4ea1-a5de-0654ae8a4e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:37.700884 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aba4a8-4b91-4ea1-a5de-0654ae8a4e92 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:37.701607 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aba4a8-4b91-4ea1-a5de-0654ae8a4e92 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:37.701918 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aba4a8-4b91-4ea1-a5de-0654ae8a4e92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:37.703363 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aba4a8-4b91-4ea1-a5de-0654ae8a4e92 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:37.720936 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 320/633] 10.222.0.27 () {56 vars in 1137 bytes} [Sun Sep 4 00:48:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:37.721731 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aba4a8-4b91-4ea1-a5de-0654ae8a4e92 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:37.731504 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86af8b4b-9fd8-485c-bd04-f14acbfb9b2a None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:37.767402 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5366a98-4251-4765-8e3b-6e755c003274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:37.769101 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5366a98-4251-4765-8e3b-6e755c003274 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:37.769649 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5366a98-4251-4765-8e3b-6e755c003274 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:37.770118 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5366a98-4251-4765-8e3b-6e755c003274 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:37.771085 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5366a98-4251-4765-8e3b-6e755c003274 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:37.771610 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5366a98-4251-4765-8e3b-6e755c003274 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:37.773292 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5366a98-4251-4765-8e3b-6e755c003274 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:37.785780 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 314/634] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:37.787391 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d423571-c90a-473d-b6c4-5e2020f9a60c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:37.787744 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d423571-c90a-473d-b6c4-5e2020f9a60c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:37.788092 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d423571-c90a-473d-b6c4-5e2020f9a60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:37.807414 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-9d423571-c90a-473d-b6c4-5e2020f9a60c None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:37.808767 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9d423571-c90a-473d-b6c4-5e2020f9a60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:37.832178 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aba4a8-4b91-4ea1-a5de-0654ae8a4e92 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:37.845985 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:37.868127 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953cb4e9-4a43-4ae6-964f-ea8dad61c4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:37.868886 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953cb4e9-4a43-4ae6-964f-ea8dad61c4e4 None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:37.869101 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953cb4e9-4a43-4ae6-964f-ea8dad61c4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:37.869373 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953cb4e9-4a43-4ae6-964f-ea8dad61c4e4 None admin] PATH_INFO: `/v3/projects/5100a8d316d243fb8f82e9044e1b3b06/users/2297d8b3fe8e47eb9fd39a6e3eea8f54/roles/6e3e398ba29e436e8532a45d2fdf1321` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:37.883520 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 315/635] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:37.884577 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953cb4e9-4a43-4ae6-964f-ea8dad61c4e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=5100a8d316d243fb8f82e9044e1b3b06, user_id=2297d8b3fe8e47eb9fd39a6e3eea8f54, role_id=6e3e398ba29e436e8532a45d2fdf1321)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:37.884977 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d423571-c90a-473d-b6c4-5e2020f9a60c None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:37.886197 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953cb4e9-4a43-4ae6-964f-ea8dad61c4e4 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:37.937097 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:37.988099 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 321/636] 10.222.0.27 () {58 vars in 1439 bytes} [Sun Sep 4 00:48:37 2022] PUT /identity/v3/projects/5100a8d316d243fb8f82e9044e1b3b06/users/2297d8b3fe8e47eb9fd39a6e3eea8f54/roles/6e3e398ba29e436e8532a45d2fdf1321 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 00:48:37.997311 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d423571-c90a-473d-b6c4-5e2020f9a60c None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:37.997539 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953cb4e9-4a43-4ae6-964f-ea8dad61c4e4 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:38.041369 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad9051-dfcf-4983-9117-2501c56fd769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:38.041913 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad9051-dfcf-4983-9117-2501c56fd769 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:38.042192 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad9051-dfcf-4983-9117-2501c56fd769 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:38.042465 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad9051-dfcf-4983-9117-2501c56fd769 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:38.126572 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad9051-dfcf-4983-9117-2501c56fd769 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:38.128536 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad9051-dfcf-4983-9117-2501c56fd769 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:38.153241 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953cb4e9-4a43-4ae6-964f-ea8dad61c4e4 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:38.180431 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:38.196822 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a1a779-ca1c-4b92-991d-a5a6fee25c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:38.197445 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1a779-ca1c-4b92-991d-a5a6fee25c7b None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:38.197724 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1a779-ca1c-4b92-991d-a5a6fee25c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:38.197993 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1a779-ca1c-4b92-991d-a5a6fee25c7b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:38.205916 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a1a779-ca1c-4b92-991d-a5a6fee25c7b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:38.207400 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a1a779-ca1c-4b92-991d-a5a6fee25c7b None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:38.219044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-53a1a779-ca1c-4b92-991d-a5a6fee25c7b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 04 00:48:38.219044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:38.219044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:38.219044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:38.219044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:38.219044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:38.219044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:38.219044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:38.219044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:38.219044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:38.219044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:38.219044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:38.219044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:48:38.219044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:48:38.219044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:48:38.219044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:48:38.220027 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:38.220027 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:38.220027 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:48:38.220027 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:38.220027 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:48:38.220027 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:38.220027 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:38.220027 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:38.220027 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:38.220027 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:38.220027 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:38.220027 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:38.220027 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:38.220027 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:38.220027 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:38.220027 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:38.220027 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:38.220027 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:38.221739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:38.221739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:38.221739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:48:38.221739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:48:38.221739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:48:38.221739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:48:38.221739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:48:38.221739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:48:38.221739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 04 00:48:38.221739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:38.221739 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 322/637] 10.222.0.27 () {56 vars in 1093 bytes} [Sun Sep 4 00:48:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:38.222417 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ce2c7-10cc-478a-b96e-41dc7a9556ed None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:38.222417 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ce2c7-10cc-478a-b96e-41dc7a9556ed None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:38.223303 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ce2c7-10cc-478a-b96e-41dc7a9556ed None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:38.224064 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 323/638] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:38.224803 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ce2c7-10cc-478a-b96e-41dc7a9556ed None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:38.249711 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:38.258354 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 316/639] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:48:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:48:38.258354 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03e64d-cac4-4aae-8e6f-1e33fff38b10 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:38.258354 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03e64d-cac4-4aae-8e6f-1e33fff38b10 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:38.258354 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03e64d-cac4-4aae-8e6f-1e33fff38b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:38.277416 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-bb03e64d-cac4-4aae-8e6f-1e33fff38b10 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:38.278615 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bb03e64d-cac4-4aae-8e6f-1e33fff38b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:38.333041 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ce2c7-10cc-478a-b96e-41dc7a9556ed None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:38.360483 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 317/640] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:38.360483 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb03e64d-cac4-4aae-8e6f-1e33fff38b10 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:38.383206 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b7929-bdc3-4b43-b70f-e3aea7ee362b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:38.383974 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b7929-bdc3-4b43-b70f-e3aea7ee362b None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:38.384325 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b7929-bdc3-4b43-b70f-e3aea7ee362b None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:38.384682 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b7929-bdc3-4b43-b70f-e3aea7ee362b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:38.397376 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b7929-bdc3-4b43-b70f-e3aea7ee362b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:38.402759 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-d25b7929-bdc3-4b43-b70f-e3aea7ee362b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:38.402759 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:38.402759 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:38.402759 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:38.402759 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:38.402759 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:38.402759 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:38.402759 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:38.402759 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:38.402759 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:38.402759 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:38.402759 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:38.402759 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:38.402759 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:38.402759 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:38.402759 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:38.403903 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:38.403903 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:38.403903 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:38.403903 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:38.403903 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:38.403903 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:38.403903 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:38.403903 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:38.403903 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:38.403903 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:38.403903 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:38.403903 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:38.403903 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:38.403903 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:38.403903 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:38.403903 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:38.405001 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:38.405001 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:38.405001 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:38.405001 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:38.405001 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:38.405001 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:38.405001 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:38.405001 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:38.405001 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:38.405001 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:38.405001 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:38.405001 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:38.411040 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 324/641] 10.222.0.27 () {56 vars in 1102 bytes} [Sun Sep 4 00:48:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:38.412099 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b7929-bdc3-4b43-b70f-e3aea7ee362b None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:38.446908 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:38.482924 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb03e64d-cac4-4aae-8e6f-1e33fff38b10 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:38.519917 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:38.521374 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e81ff2-6957-46ac-8adc-8f5f2e517888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:38.521991 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e81ff2-6957-46ac-8adc-8f5f2e517888 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:38.522275 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e81ff2-6957-46ac-8adc-8f5f2e517888 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:38.522552 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e81ff2-6957-46ac-8adc-8f5f2e517888 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:38.523292 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e81ff2-6957-46ac-8adc-8f5f2e517888 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:38.523599 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e81ff2-6957-46ac-8adc-8f5f2e517888 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:38.525076 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e81ff2-6957-46ac-8adc-8f5f2e517888 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:38.543501 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 318/642] 10.222.0.27 () {56 vars in 1133 bytes} [Sun Sep 4 00:48:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:38.544335 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e81ff2-6957-46ac-8adc-8f5f2e517888 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:38.544925 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b7929-bdc3-4b43-b70f-e3aea7ee362b None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:38.587809 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab71fa-3d5f-42e9-bb87-fbc3b29c1db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:38.589044 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab71fa-3d5f-42e9-bb87-fbc3b29c1db1 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:38.589534 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab71fa-3d5f-42e9-bb87-fbc3b29c1db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:38.590908 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab71fa-3d5f-42e9-bb87-fbc3b29c1db1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:38.604416 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab71fa-3d5f-42e9-bb87-fbc3b29c1db1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:38.605961 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab71fa-3d5f-42e9-bb87-fbc3b29c1db1 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:38.624662 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-14ab71fa-3d5f-42e9-bb87-fbc3b29c1db1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:38.624662 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:38.624662 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:38.624662 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:38.624662 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:38.624662 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:38.624662 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:38.624662 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:38.624662 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:38.624662 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:38.624662 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:38.624662 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:38.624662 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:38.624662 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:38.624662 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:38.624662 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:38.625711 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:38.625711 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:38.625711 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:38.625711 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:38.625711 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:38.625711 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:38.625711 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:38.625711 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:38.625711 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:38.625711 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:38.625711 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:38.625711 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:38.625711 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:38.625711 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:38.625711 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:38.625711 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:38.625711 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:38.627211 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:38.627211 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:38.627211 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:38.627211 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 325/643] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:38.627562 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab71fa-3d5f-42e9-bb87-fbc3b29c1db1 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:38.676588 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e81ff2-6957-46ac-8adc-8f5f2e517888 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:38.704068 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:38.712750 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6edc3d-7efd-43bf-9e92-46e5a76dfd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:38.715185 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6edc3d-7efd-43bf-9e92-46e5a76dfd1e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:38.715185 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6edc3d-7efd-43bf-9e92-46e5a76dfd1e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:38.715185 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6edc3d-7efd-43bf-9e92-46e5a76dfd1e None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:38.715185 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6edc3d-7efd-43bf-9e92-46e5a76dfd1e None admin] RBAC: Adding query filter params (name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:38.715185 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6edc3d-7efd-43bf-9e92-46e5a76dfd1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:38.716129 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6edc3d-7efd-43bf-9e92-46e5a76dfd1e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:38.724408 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 319/644] 10.222.0.27 () {56 vars in 1103 bytes} [Sun Sep 4 00:48:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:38.725334 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6edc3d-7efd-43bf-9e92-46e5a76dfd1e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:38.751128 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab71fa-3d5f-42e9-bb87-fbc3b29c1db1 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:38.787595 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2debf089-1cb7-4b45-ac13-18b1ab766138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:38.788209 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2debf089-1cb7-4b45-ac13-18b1ab766138 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:38.788517 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2debf089-1cb7-4b45-ac13-18b1ab766138 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:38.788802 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2debf089-1cb7-4b45-ac13-18b1ab766138 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:38.791755 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:38.828331 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6edc3d-7efd-43bf-9e92-46e5a76dfd1e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:38.862414 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a44766-2276-4bd1-86b9-06c7022fdc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:38.862938 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a44766-2276-4bd1-86b9-06c7022fdc7c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:38.863224 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a44766-2276-4bd1-86b9-06c7022fdc7c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:38.863498 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a44766-2276-4bd1-86b9-06c7022fdc7c None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:38.864134 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a44766-2276-4bd1-86b9-06c7022fdc7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:38.864440 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a44766-2276-4bd1-86b9-06c7022fdc7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:38.865834 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a44766-2276-4bd1-86b9-06c7022fdc7c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:38.866315 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2debf089-1cb7-4b45-ac13-18b1ab766138 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:38.867781 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2debf089-1cb7-4b45-ac13-18b1ab766138 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:38.878606 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 320/645] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:38.879429 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a44766-2276-4bd1-86b9-06c7022fdc7c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:38.932876 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:38.981217 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 326/646] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:48:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:48:38.981217 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2debf089-1cb7-4b45-ac13-18b1ab766138 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:38.991541 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a44766-2276-4bd1-86b9-06c7022fdc7c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:39.015237 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:39.028679 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb3ecc-3771-45b1-a187-058397a7e581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:39.029314 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb3ecc-3771-45b1-a187-058397a7e581 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:39.029595 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb3ecc-3771-45b1-a187-058397a7e581 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:39.029867 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb3ecc-3771-45b1-a187-058397a7e581 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:39.041092 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb3ecc-3771-45b1-a187-058397a7e581 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:39.046313 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-72fb3ecc-3771-45b1-a187-058397a7e581 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:39.046313 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:39.046313 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:39.046313 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:39.046313 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:39.046313 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:39.046313 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:39.046313 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:39.046313 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:39.046313 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:39.046313 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:39.046313 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:39.046313 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:39.046313 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:39.046313 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:39.046313 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:39.047535 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:39.047535 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:39.047535 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:39.047535 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:39.047535 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:39.047535 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:39.047535 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:39.047535 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:39.047535 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:39.047535 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:39.047535 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:39.047535 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:39.047535 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:39.047535 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:39.047535 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:39.047535 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:39.049071 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:39.049071 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:39.049071 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:39.049071 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:39.049071 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:39.049071 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:39.049071 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:39.049071 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:39.049071 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:39.049071 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:39.049071 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:39.049071 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:39.052784 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 321/647] 10.222.0.27 () {56 vars in 1102 bytes} [Sun Sep 4 00:48:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:39.053556 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb3ecc-3771-45b1-a187-058397a7e581 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:39.089297 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2debf089-1cb7-4b45-ac13-18b1ab766138 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:39.127670 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd23d120-4516-43d0-8eac-31f0a01004bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:39.128426 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd23d120-4516-43d0-8eac-31f0a01004bb None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:39.128739 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd23d120-4516-43d0-8eac-31f0a01004bb None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:39.129013 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd23d120-4516-43d0-8eac-31f0a01004bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:39.140012 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd23d120-4516-43d0-8eac-31f0a01004bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:39.141492 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd23d120-4516-43d0-8eac-31f0a01004bb None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:39.156377 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-fd23d120-4516-43d0-8eac-31f0a01004bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:39.156377 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:39.156377 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:39.156377 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:39.156377 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:39.156377 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:39.156377 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:39.156377 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:39.156377 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:39.156377 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:39.156377 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:39.156377 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:39.156377 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:39.156377 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:39.156377 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:39.156377 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:39.157338 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:39.157338 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:39.157338 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:39.157338 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:39.157338 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:39.157338 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:39.157338 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:39.157338 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:39.157338 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:39.157338 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:39.157338 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:39.157338 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:39.157338 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:39.157338 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:39.157338 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:39.157338 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:39.157338 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:39.163139 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:39.163139 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:39.163139 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:39.163311 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb3ecc-3771-45b1-a187-058397a7e581 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:39.163385 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 327/648] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:39.163385 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd23d120-4516-43d0-8eac-31f0a01004bb None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:39.174220 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:39.199361 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f261a29f-2ebf-4717-a131-790a42fa90da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:39.199980 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261a29f-2ebf-4717-a131-790a42fa90da None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:39.200258 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261a29f-2ebf-4717-a131-790a42fa90da None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:39.200568 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261a29f-2ebf-4717-a131-790a42fa90da None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:39.209776 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f261a29f-2ebf-4717-a131-790a42fa90da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:39.211624 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f261a29f-2ebf-4717-a131-790a42fa90da None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:39.223398 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-f261a29f-2ebf-4717-a131-790a42fa90da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:39.223398 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:39.223398 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:39.223398 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:39.223398 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:39.223398 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:39.223398 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:39.223398 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:39.223398 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:39.223398 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:39.223398 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:39.223398 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:39.223398 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:48:39.223398 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:48:39.223398 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:48:39.224296 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:48:39.224296 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:39.224296 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:39.224296 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:39.224296 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:39.224296 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:39.224296 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:39.224296 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:39.224296 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:39.224296 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:39.224296 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:39.224296 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:39.224296 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:39.224296 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:39.224296 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:39.224296 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:39.224296 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:39.225230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:48:39.225230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:48:39.225230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:48:39.225230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:48:39.225230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:48:39.225230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:48:39.225230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:39.225230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:39.225230 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 322/649] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:48:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:39.225745 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f261a29f-2ebf-4717-a131-790a42fa90da None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:39.264297 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:39.279394 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd23d120-4516-43d0-8eac-31f0a01004bb None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:39.318233 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e66c8f-492a-4aee-bb1c-f947962a91c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:39.318866 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e66c8f-492a-4aee-bb1c-f947962a91c7 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:39.319150 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e66c8f-492a-4aee-bb1c-f947962a91c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:39.319434 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e66c8f-492a-4aee-bb1c-f947962a91c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:39.329450 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e66c8f-492a-4aee-bb1c-f947962a91c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:39.331340 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e66c8f-492a-4aee-bb1c-f947962a91c7 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:39.332802 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f261a29f-2ebf-4717-a131-790a42fa90da None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:39.349653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-65e66c8f-492a-4aee-bb1c-f947962a91c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:39.349653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:39.349653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:39.349653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:39.349653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:39.349653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:39.349653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:39.349653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:39.349653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:39.349653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:39.349653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:39.349653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:39.349653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:39.349653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:39.349653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:39.349653 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:39.350959 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:39.350959 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:39.350959 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:39.350959 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:39.350959 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:39.350959 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:39.350959 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:39.350959 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:39.350959 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:39.350959 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:39.350959 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:39.350959 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:39.350959 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:39.350959 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:39.350959 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:39.350959 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:39.350959 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:39.351881 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:39.351881 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:39.351881 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:39.351881 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 328/650] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:39.351881 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e66c8f-492a-4aee-bb1c-f947962a91c7 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:39.372424 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f940a2-253c-4583-881f-a9d30ed1d889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:39.373049 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f940a2-253c-4583-881f-a9d30ed1d889 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:39.373330 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f940a2-253c-4583-881f-a9d30ed1d889 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:39.373603 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f940a2-253c-4583-881f-a9d30ed1d889 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:39.374260 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f940a2-253c-4583-881f-a9d30ed1d889 None admin] RBAC: Adding query filter params (name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:39.374563 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f940a2-253c-4583-881f-a9d30ed1d889 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:39.375920 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f940a2-253c-4583-881f-a9d30ed1d889 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:39.384787 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 323/651] 10.222.0.27 () {56 vars in 1103 bytes} [Sun Sep 4 00:48:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:39.390372 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f940a2-253c-4583-881f-a9d30ed1d889 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:39.423806 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:39.467764 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e66c8f-492a-4aee-bb1c-f947962a91c7 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:39.499330 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f940a2-253c-4583-881f-a9d30ed1d889 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:39.506723 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403d1f7-c417-429f-9275-e55d7f8d3a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:39.507751 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403d1f7-c417-429f-9275-e55d7f8d3a85 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:39.507751 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403d1f7-c417-429f-9275-e55d7f8d3a85 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:39.508107 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403d1f7-c417-429f-9275-e55d7f8d3a85 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:39.508846 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8403d1f7-c417-429f-9275-e55d7f8d3a85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:39.509214 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8403d1f7-c417-429f-9275-e55d7f8d3a85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:39.510648 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8403d1f7-c417-429f-9275-e55d7f8d3a85 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:39.519185 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:39.524415 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 329/652] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:39.529389 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403d1f7-c417-429f-9275-e55d7f8d3a85 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:39.537933 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c568cb-83bd-4888-9e6c-1805dd59eefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:39.539067 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c568cb-83bd-4888-9e6c-1805dd59eefb None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:39.539418 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c568cb-83bd-4888-9e6c-1805dd59eefb None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:39.540005 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c568cb-83bd-4888-9e6c-1805dd59eefb None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:39.540802 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c568cb-83bd-4888-9e6c-1805dd59eefb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:39.541178 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c568cb-83bd-4888-9e6c-1805dd59eefb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:39.542652 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c568cb-83bd-4888-9e6c-1805dd59eefb None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:39.559458 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 324/653] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:48:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:39.559458 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c568cb-83bd-4888-9e6c-1805dd59eefb None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:39.639156 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403d1f7-c417-429f-9275-e55d7f8d3a85 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:39.670999 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:39.678045 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776fa6e3-4e8b-444c-91a0-c3c96cd7d6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:39.678648 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fa6e3-4e8b-444c-91a0-c3c96cd7d6b8 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:39.678926 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fa6e3-4e8b-444c-91a0-c3c96cd7d6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:39.679198 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fa6e3-4e8b-444c-91a0-c3c96cd7d6b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:39.679840 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776fa6e3-4e8b-444c-91a0-c3c96cd7d6b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:39.680140 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776fa6e3-4e8b-444c-91a0-c3c96cd7d6b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:39.680324 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c568cb-83bd-4888-9e6c-1805dd59eefb None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:39.681541 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776fa6e3-4e8b-444c-91a0-c3c96cd7d6b8 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:39.693661 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 330/654] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:39.694329 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776fa6e3-4e8b-444c-91a0-c3c96cd7d6b8 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:39.724316 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1918b2-cca8-4226-80f9-36082523785e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:39.724978 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1918b2-cca8-4226-80f9-36082523785e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:39.725253 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1918b2-cca8-4226-80f9-36082523785e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:39.725522 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1918b2-cca8-4226-80f9-36082523785e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:39.736509 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1918b2-cca8-4226-80f9-36082523785e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:39.737954 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1918b2-cca8-4226-80f9-36082523785e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:39.753033 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-1e1918b2-cca8-4226-80f9-36082523785e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:39.753033 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:39.753033 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:39.753033 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:39.753033 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:39.753033 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:39.753033 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:39.753033 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:39.753033 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:39.753033 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:39.753033 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:39.753033 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:39.753033 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:39.753033 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:39.753033 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:39.753033 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:39.753972 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:39.753972 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:39.753972 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:39.753972 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:39.753972 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:39.753972 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:39.753972 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:39.753972 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:39.753972 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:39.753972 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:39.753972 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:39.753972 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:39.753972 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:39.753972 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:39.753972 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:39.753972 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:39.753972 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:39.754849 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:39.754849 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:39.754849 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:39.754849 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 325/655] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:39.754849 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1918b2-cca8-4226-80f9-36082523785e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:39.762907 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:39.809016 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776fa6e3-4e8b-444c-91a0-c3c96cd7d6b8 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:39.844189 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b07cd-f3f7-4e64-b390-c12626c1c8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:39.844859 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b07cd-f3f7-4e64-b390-c12626c1c8bb None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:39.845126 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b07cd-f3f7-4e64-b390-c12626c1c8bb None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:39.846093 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b07cd-f3f7-4e64-b390-c12626c1c8bb None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:39.853708 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2b07cd-f3f7-4e64-b390-c12626c1c8bb None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:39.855153 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2b07cd-f3f7-4e64-b390-c12626c1c8bb None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:39.862926 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1918b2-cca8-4226-80f9-36082523785e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:39.866380 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-0d2b07cd-f3f7-4e64-b390-c12626c1c8bb None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 04 00:48:39.866380 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:39.866380 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:39.866380 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:39.866380 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:39.866380 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:39.866380 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:39.866380 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:39.866380 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:39.866380 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:39.866380 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:39.866380 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:39.866380 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:48:39.866380 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:48:39.866380 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:48:39.866380 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:48:39.867277 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:39.867277 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:39.867277 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:48:39.867277 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:39.867277 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:48:39.867277 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:39.867277 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:39.867277 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:39.867277 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:39.867277 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:39.867277 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:39.867277 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:39.867277 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:39.867277 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:39.867277 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:39.867277 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:39.867277 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:39.867277 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:39.873087 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:39.873087 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:39.873087 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:48:39.873087 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:48:39.873087 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:48:39.873087 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:48:39.873087 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:48:39.873087 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:48:39.873087 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 04 00:48:39.873087 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:39.873087 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 331/656] 10.222.0.27 () {56 vars in 1102 bytes} [Sun Sep 4 00:48:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:39.873087 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b07cd-f3f7-4e64-b390-c12626c1c8bb None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:39.904875 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a34e5-fa70-4f66-a7ae-336b79455199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:39.906221 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a34e5-fa70-4f66-a7ae-336b79455199 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:39.906592 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a34e5-fa70-4f66-a7ae-336b79455199 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:39.906995 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a34e5-fa70-4f66-a7ae-336b79455199 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:39.908311 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1a34e5-fa70-4f66-a7ae-336b79455199 None admin] RBAC: Adding query filter params (role.id=6e3e398ba29e436e8532a45d2fdf1321, scope.project.id=5100a8d316d243fb8f82e9044e1b3b06, user.id=1179503bdab547c99175a27b7ce27b9d) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:39.908667 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1a34e5-fa70-4f66-a7ae-336b79455199 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:39.910079 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1a34e5-fa70-4f66-a7ae-336b79455199 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:39.924641 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 326/657] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:48:39 2022] GET /identity/v3/role_assignments?user.id=1179503bdab547c99175a27b7ce27b9d&scope.project.id=5100a8d316d243fb8f82e9044e1b3b06&role.id=6e3e398ba29e436e8532a45d2fdf1321 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:39.924641 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a34e5-fa70-4f66-a7ae-336b79455199 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:39.941032 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:39.991488 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b07cd-f3f7-4e64-b390-c12626c1c8bb None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:40.009900 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:40.029393 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b730e0-03d7-4f7a-92ca-4cfaef4470c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:40.030144 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b730e0-03d7-4f7a-92ca-4cfaef4470c8 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:40.030489 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b730e0-03d7-4f7a-92ca-4cfaef4470c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:40.030855 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b730e0-03d7-4f7a-92ca-4cfaef4470c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:40.038599 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b730e0-03d7-4f7a-92ca-4cfaef4470c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:40.040727 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b730e0-03d7-4f7a-92ca-4cfaef4470c8 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:40.048183 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a34e5-fa70-4f66-a7ae-336b79455199 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:40.054528 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-01b730e0-03d7-4f7a-92ca-4cfaef4470c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:40.054528 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:40.054528 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:40.054528 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:40.054528 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:40.054528 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:40.054528 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:40.054528 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:40.054528 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:40.054528 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:40.054528 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:40.054528 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:40.054528 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:48:40.054528 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:48:40.054528 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:48:40.055643 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:48:40.055643 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:40.055643 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:40.055643 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:40.055643 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:40.055643 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:40.055643 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:40.055643 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:40.055643 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:40.055643 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:40.055643 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:40.055643 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:40.055643 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:40.055643 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:40.055643 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:40.055643 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:40.055643 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:40.056821 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:48:40.056821 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:48:40.056821 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:48:40.056821 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:48:40.056821 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:48:40.056821 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:48:40.056821 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:40.056821 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:40.058307 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 332/658] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:48:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:40.059233 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b730e0-03d7-4f7a-92ca-4cfaef4470c8 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:40.091249 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3058427a-ff0a-4ee4-822c-28318db2a749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:40.091861 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058427a-ff0a-4ee4-822c-28318db2a749 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:40.092143 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058427a-ff0a-4ee4-822c-28318db2a749 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:40.092416 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058427a-ff0a-4ee4-822c-28318db2a749 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:40.093152 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3058427a-ff0a-4ee4-822c-28318db2a749 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:40.093453 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3058427a-ff0a-4ee4-822c-28318db2a749 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:40.094825 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3058427a-ff0a-4ee4-822c-28318db2a749 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:40.116959 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 327/659] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:40.116959 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91740c-66d1-441f-b7d7-6ed70f73757d None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:40.116959 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91740c-66d1-441f-b7d7-6ed70f73757d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:40.116959 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91740c-66d1-441f-b7d7-6ed70f73757d None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:40.116959 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 328/660] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:40.116959 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91740c-66d1-441f-b7d7-6ed70f73757d None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:40.177934 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b730e0-03d7-4f7a-92ca-4cfaef4470c8 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:40.193880 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:40.214958 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05530573-21a3-4d12-bac6-bcbb87da1190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:40.215558 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05530573-21a3-4d12-bac6-bcbb87da1190 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:40.215837 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05530573-21a3-4d12-bac6-bcbb87da1190 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:40.216107 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05530573-21a3-4d12-bac6-bcbb87da1190 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:40.216860 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05530573-21a3-4d12-bac6-bcbb87da1190 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:40.217166 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05530573-21a3-4d12-bac6-bcbb87da1190 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:40.218666 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05530573-21a3-4d12-bac6-bcbb87da1190 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:40.231258 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91740c-66d1-441f-b7d7-6ed70f73757d None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:40.245643 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 333/661] 10.222.0.27 () {56 vars in 1139 bytes} [Sun Sep 4 00:48:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:40.250714 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05530573-21a3-4d12-bac6-bcbb87da1190 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:40.268980 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7119653-5d2e-41e8-b805-5552c654e62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:40.269592 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7119653-5d2e-41e8-b805-5552c654e62e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:40.269936 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7119653-5d2e-41e8-b805-5552c654e62e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:40.270224 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7119653-5d2e-41e8-b805-5552c654e62e None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:40.270917 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7119653-5d2e-41e8-b805-5552c654e62e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:40.271229 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7119653-5d2e-41e8-b805-5552c654e62e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:40.272683 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7119653-5d2e-41e8-b805-5552c654e62e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:40.290108 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 329/662] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:48:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:40.290108 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd20ef1-d424-457a-ba68-cc860002ffe6 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:40.290108 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd20ef1-d424-457a-ba68-cc860002ffe6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:40.290108 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd20ef1-d424-457a-ba68-cc860002ffe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:40.295741 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:40.306267 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-bcd20ef1-d424-457a-ba68-cc860002ffe6 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:40.309254 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bcd20ef1-d424-457a-ba68-cc860002ffe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:40.366463 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05530573-21a3-4d12-bac6-bcbb87da1190 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:40.384492 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 330/663] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:40.384492 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd20ef1-d424-457a-ba68-cc860002ffe6 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:40.413612 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d42d703-a2a2-4a53-95dc-cd544df259a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:40.414398 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42d703-a2a2-4a53-95dc-cd544df259a1 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:40.415039 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42d703-a2a2-4a53-95dc-cd544df259a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:40.415457 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42d703-a2a2-4a53-95dc-cd544df259a1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:40.424085 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d42d703-a2a2-4a53-95dc-cd544df259a1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:40.425765 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d42d703-a2a2-4a53-95dc-cd544df259a1 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:40.439909 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-2d42d703-a2a2-4a53-95dc-cd544df259a1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 04 00:48:40.439909 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:40.439909 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:40.439909 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:40.439909 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:40.439909 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:40.439909 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:40.439909 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:40.439909 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:40.439909 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:40.439909 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:40.439909 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:40.439909 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:48:40.439909 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:48:40.439909 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:48:40.439909 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:48:40.441133 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:40.441133 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:40.441133 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:48:40.441133 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:40.441133 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:48:40.441133 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:40.441133 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:40.441133 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:40.441133 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:40.441133 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:40.441133 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:40.441133 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:40.441133 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:40.441133 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:40.441133 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:40.441133 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:40.441133 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:40.441133 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:40.442310 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:40.442310 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:40.442310 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:48:40.442310 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:48:40.442310 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:48:40.442310 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:48:40.442310 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:48:40.442310 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:48:40.442310 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 04 00:48:40.442310 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:40.444807 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 334/664] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:48:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:40.445207 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d42d703-a2a2-4a53-95dc-cd544df259a1 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:40.448828 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:40.499302 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd20ef1-d424-457a-ba68-cc860002ffe6 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:40.537697 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd29984-90bb-43a2-9ed5-41cc20ec5964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:40.538703 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd29984-90bb-43a2-9ed5-41cc20ec5964 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:40.539237 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd29984-90bb-43a2-9ed5-41cc20ec5964 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:40.539714 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd29984-90bb-43a2-9ed5-41cc20ec5964 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:40.563142 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:40.569998 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d42d703-a2a2-4a53-95dc-cd544df259a1 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:40.609614 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713a83d-373c-4112-abe9-669d6f34907e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:40.610635 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713a83d-373c-4112-abe9-669d6f34907e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:40.611294 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713a83d-373c-4112-abe9-669d6f34907e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:40.611773 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713a83d-373c-4112-abe9-669d6f34907e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:40.612837 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713a83d-373c-4112-abe9-669d6f34907e None admin] RBAC: Adding query filter params (role.id=6e3e398ba29e436e8532a45d2fdf1321, scope.project.id=5100a8d316d243fb8f82e9044e1b3b06, user.id=68bc00bda5194cd8a7928f8c4df4ccd1) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:40.613377 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713a83d-373c-4112-abe9-669d6f34907e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:40.615245 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713a83d-373c-4112-abe9-669d6f34907e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:40.621667 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd29984-90bb-43a2-9ed5-41cc20ec5964 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:40.622390 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 335/665] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:48:40 2022] GET /identity/v3/role_assignments?user.id=68bc00bda5194cd8a7928f8c4df4ccd1&scope.project.id=5100a8d316d243fb8f82e9044e1b3b06&role.id=6e3e398ba29e436e8532a45d2fdf1321 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:40.623170 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd29984-90bb-43a2-9ed5-41cc20ec5964 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:40.623828 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713a83d-373c-4112-abe9-669d6f34907e None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:40.688434 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:40.738218 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 331/666] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:48:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:48:40.739033 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd29984-90bb-43a2-9ed5-41cc20ec5964 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:40.742613 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713a83d-373c-4112-abe9-669d6f34907e None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:40.785040 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b27a3d-4990-41fe-9e5f-025a7fd724fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:40.786043 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b27a3d-4990-41fe-9e5f-025a7fd724fc None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:40.786531 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b27a3d-4990-41fe-9e5f-025a7fd724fc None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:40.787010 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b27a3d-4990-41fe-9e5f-025a7fd724fc None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:40.787987 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b27a3d-4990-41fe-9e5f-025a7fd724fc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:40.788556 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b27a3d-4990-41fe-9e5f-025a7fd724fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:40.790637 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b27a3d-4990-41fe-9e5f-025a7fd724fc None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:40.811638 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 336/667] 10.222.0.27 () {56 vars in 1137 bytes} [Sun Sep 4 00:48:40 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:40.812929 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b27a3d-4990-41fe-9e5f-025a7fd724fc None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:40.837714 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:40.856264 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd29984-90bb-43a2-9ed5-41cc20ec5964 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:40.898646 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73578c-531d-43d3-b2e9-4d9ef857018a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:40.899644 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73578c-531d-43d3-b2e9-4d9ef857018a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:40.900159 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73578c-531d-43d3-b2e9-4d9ef857018a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:40.900594 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73578c-531d-43d3-b2e9-4d9ef857018a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:40.915336 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a73578c-531d-43d3-b2e9-4d9ef857018a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:40.920345 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-3a73578c-531d-43d3-b2e9-4d9ef857018a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:40.920345 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:40.920345 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:40.920345 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:40.920345 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:40.920345 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:40.920345 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:40.920345 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:40.920345 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:40.920345 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:40.920345 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:40.920345 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:40.920345 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:40.920345 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:40.920345 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:40.920345 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:40.921298 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:40.921298 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:40.921298 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:40.921298 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:40.921298 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:40.921298 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:40.921298 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:40.921298 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:40.921298 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:40.921298 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:40.921298 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:40.921298 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:40.921298 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:40.921298 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:40.921298 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:40.921298 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:40.922334 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:40.922334 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:40.922334 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:40.922334 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:40.922334 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:40.922334 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:40.922334 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:40.922334 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:40.922334 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:40.922334 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:40.922334 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:40.922334 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:40.924727 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 332/668] 10.222.0.27 () {56 vars in 1102 bytes} [Sun Sep 4 00:48:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:40.925051 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73578c-531d-43d3-b2e9-4d9ef857018a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:40.948972 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b27a3d-4990-41fe-9e5f-025a7fd724fc None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:40.952724 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:40.996172 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373046e7-8a84-4274-b452-e1410745841a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:40.997119 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373046e7-8a84-4274-b452-e1410745841a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:40.997668 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373046e7-8a84-4274-b452-e1410745841a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:40.998115 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373046e7-8a84-4274-b452-e1410745841a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:41.009886 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373046e7-8a84-4274-b452-e1410745841a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:41.011810 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373046e7-8a84-4274-b452-e1410745841a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:41.028569 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-373046e7-8a84-4274-b452-e1410745841a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:41.028569 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:41.028569 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:41.028569 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:41.028569 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:41.028569 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:41.028569 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:41.028569 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:41.028569 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:41.028569 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:41.028569 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:41.028569 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:41.028569 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:41.028569 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:41.028569 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:41.028569 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:41.029595 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:41.029595 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:41.029595 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:41.029595 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:41.029595 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:41.029595 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:41.029595 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:41.029595 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:41.029595 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:41.029595 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:41.029595 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:41.029595 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:41.029595 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:41.029595 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:41.029595 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:41.029595 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:41.029595 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:41.030458 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:41.030458 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:41.030458 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:41.032067 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 337/669] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:41.033836 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373046e7-8a84-4274-b452-e1410745841a None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:41.054335 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73578c-531d-43d3-b2e9-4d9ef857018a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:41.102281 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810b43b-404b-440a-a38d-89aa2f129227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:41.103184 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810b43b-404b-440a-a38d-89aa2f129227 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:41.103472 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810b43b-404b-440a-a38d-89aa2f129227 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:41.103751 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810b43b-404b-440a-a38d-89aa2f129227 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:41.114326 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:41.148376 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373046e7-8a84-4274-b452-e1410745841a None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:41.188708 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fdda0d-fb09-490e-b8cd-0957ec86a4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:41.189803 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdda0d-fb09-490e-b8cd-0957ec86a4a7 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:41.190223 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdda0d-fb09-490e-b8cd-0957ec86a4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:41.190635 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdda0d-fb09-490e-b8cd-0957ec86a4a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:41.191432 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fdda0d-fb09-490e-b8cd-0957ec86a4a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:41.191874 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fdda0d-fb09-490e-b8cd-0957ec86a4a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:41.192763 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2810b43b-404b-440a-a38d-89aa2f129227 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:41.193578 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fdda0d-fb09-490e-b8cd-0957ec86a4a7 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:41.194215 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2810b43b-404b-440a-a38d-89aa2f129227 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:41.207469 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 338/670] 10.222.0.27 () {56 vars in 1103 bytes} [Sun Sep 4 00:48:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:41.208583 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fdda0d-fb09-490e-b8cd-0957ec86a4a7 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:41.216491 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:41.317241 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 333/671] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:48:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:48:41.318788 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810b43b-404b-440a-a38d-89aa2f129227 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:41.333036 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fdda0d-fb09-490e-b8cd-0957ec86a4a7 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:41.364510 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:41.370926 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31c46e-feb2-4edc-bee3-208d1318d416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:41.377117 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31c46e-feb2-4edc-bee3-208d1318d416 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:41.377117 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31c46e-feb2-4edc-bee3-208d1318d416 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:41.377117 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31c46e-feb2-4edc-bee3-208d1318d416 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:41.377117 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae31c46e-feb2-4edc-bee3-208d1318d416 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:41.377117 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae31c46e-feb2-4edc-bee3-208d1318d416 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:41.377117 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae31c46e-feb2-4edc-bee3-208d1318d416 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:41.388592 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 339/672] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:41.389847 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31c46e-feb2-4edc-bee3-208d1318d416 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:41.440392 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810b43b-404b-440a-a38d-89aa2f129227 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:41.458922 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:41.476842 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d5c92-ae3c-4058-848b-e4ac946fa081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:41.477475 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d5c92-ae3c-4058-848b-e4ac946fa081 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:41.477755 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d5c92-ae3c-4058-848b-e4ac946fa081 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:41.478040 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d5c92-ae3c-4058-848b-e4ac946fa081 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:41.490343 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5d5c92-ae3c-4058-848b-e4ac946fa081 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:41.491821 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5d5c92-ae3c-4058-848b-e4ac946fa081 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:41.498889 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31c46e-feb2-4edc-bee3-208d1318d416 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:41.508289 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-6f5d5c92-ae3c-4058-848b-e4ac946fa081 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:41.508289 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:41.508289 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:41.508289 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:41.508289 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:41.508289 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:41.508289 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:41.508289 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:41.508289 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:41.508289 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:41.508289 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:41.508289 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:41.508289 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:41.508289 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:41.508289 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:41.508289 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:41.509446 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:41.509446 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:41.509446 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:41.509446 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:41.509446 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:41.509446 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:41.509446 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:41.509446 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:41.509446 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:41.509446 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:41.509446 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:41.509446 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:41.509446 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:41.509446 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:41.509446 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:41.509446 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:41.509446 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:41.510502 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:41.510502 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:41.510502 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:41.514254 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 334/673] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:41.515097 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d5c92-ae3c-4058-848b-e4ac946fa081 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:41.539206 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d7fd74-4d09-4bc2-9819-f3c3df1a0f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:41.540394 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7fd74-4d09-4bc2-9819-f3c3df1a0f9b None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:41.540876 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7fd74-4d09-4bc2-9819-f3c3df1a0f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:41.541300 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7fd74-4d09-4bc2-9819-f3c3df1a0f9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:41.552284 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d7fd74-4d09-4bc2-9819-f3c3df1a0f9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:41.553977 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d7fd74-4d09-4bc2-9819-f3c3df1a0f9b None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:41.569830 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-74d7fd74-4d09-4bc2-9819-f3c3df1a0f9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:41.569830 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:41.569830 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:41.569830 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:41.569830 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:41.569830 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:41.569830 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:41.569830 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:41.569830 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:41.569830 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:41.569830 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:41.569830 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:41.569830 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:41.569830 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:41.569830 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:41.569830 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:41.570982 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:41.570982 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:41.570982 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:41.570982 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:41.570982 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:41.570982 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:41.570982 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:41.570982 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:41.570982 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:41.570982 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:41.570982 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:41.570982 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:41.570982 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:41.570982 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:41.570982 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:41.570982 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:41.570982 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:41.572007 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:41.572007 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:41.572007 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:41.573668 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 340/674] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:41.574639 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d7fd74-4d09-4bc2-9819-f3c3df1a0f9b None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:41.623328 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d5c92-ae3c-4058-848b-e4ac946fa081 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:41.623923 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:41.660506 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127eeb6-27c0-411a-9644-58e02acd8be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:41.660955 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127eeb6-27c0-411a-9644-58e02acd8be7 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:41.661243 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127eeb6-27c0-411a-9644-58e02acd8be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:41.661526 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127eeb6-27c0-411a-9644-58e02acd8be7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:41.669741 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6127eeb6-27c0-411a-9644-58e02acd8be7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:41.671194 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6127eeb6-27c0-411a-9644-58e02acd8be7 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:41.683898 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-6127eeb6-27c0-411a-9644-58e02acd8be7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:41.683898 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:41.683898 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:41.683898 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:41.683898 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:41.683898 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:41.683898 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:41.683898 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:41.683898 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:41.683898 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:41.683898 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:41.683898 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:41.683898 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:48:41.683898 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:48:41.683898 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:48:41.685024 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:48:41.685024 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:41.685024 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:41.685024 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:41.685024 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:41.685024 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:41.685024 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:41.685024 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:41.685024 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:41.685024 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:41.685024 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:41.685024 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:41.685024 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:41.685024 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:41.685024 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:41.685024 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:41.685024 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:41.686042 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:48:41.686042 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:48:41.686042 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:48:41.686042 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:48:41.686042 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:48:41.686042 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:48:41.686042 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:41.686042 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:41.686042 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 335/675] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:48:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:41.686515 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127eeb6-27c0-411a-9644-58e02acd8be7 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:41.687257 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d7fd74-4d09-4bc2-9819-f3c3df1a0f9b None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:41.720552 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:41.724533 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b257e9f-c4d1-4657-8411-f4818dd3c465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:41.725322 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b257e9f-c4d1-4657-8411-f4818dd3c465 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:41.725744 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b257e9f-c4d1-4657-8411-f4818dd3c465 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:41.726153 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b257e9f-c4d1-4657-8411-f4818dd3c465 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:41.727044 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b257e9f-c4d1-4657-8411-f4818dd3c465 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:41.727544 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b257e9f-c4d1-4657-8411-f4818dd3c465 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:41.729131 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b257e9f-c4d1-4657-8411-f4818dd3c465 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:41.742171 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 341/676] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:41.748487 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b257e9f-c4d1-4657-8411-f4818dd3c465 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:41.805028 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127eeb6-27c0-411a-9644-58e02acd8be7 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:41.843737 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d659898-b88e-47b8-aaca-8f9f16feeda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:41.844268 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d659898-b88e-47b8-aaca-8f9f16feeda8 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:41.844600 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d659898-b88e-47b8-aaca-8f9f16feeda8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:41.844881 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d659898-b88e-47b8-aaca-8f9f16feeda8 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:41.845670 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d659898-b88e-47b8-aaca-8f9f16feeda8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:41.845992 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d659898-b88e-47b8-aaca-8f9f16feeda8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:41.847456 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d659898-b88e-47b8-aaca-8f9f16feeda8 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:41.850676 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b257e9f-c4d1-4657-8411-f4818dd3c465 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:41.853209 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:41.861964 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 336/677] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:41.863245 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d659898-b88e-47b8-aaca-8f9f16feeda8 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:41.887556 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce297ae5-2d2a-4cc9-b715-79bf99b81200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:41.888161 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce297ae5-2d2a-4cc9-b715-79bf99b81200 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:41.888438 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce297ae5-2d2a-4cc9-b715-79bf99b81200 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:41.888990 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce297ae5-2d2a-4cc9-b715-79bf99b81200 None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:41.889691 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce297ae5-2d2a-4cc9-b715-79bf99b81200 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:41.890047 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce297ae5-2d2a-4cc9-b715-79bf99b81200 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:41.891482 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce297ae5-2d2a-4cc9-b715-79bf99b81200 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:41.904864 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 342/678] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:48:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:41.905879 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce297ae5-2d2a-4cc9-b715-79bf99b81200 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:41.968450 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:41.979725 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d659898-b88e-47b8-aaca-8f9f16feeda8 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:42.012264 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce297ae5-2d2a-4cc9-b715-79bf99b81200 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:42.019599 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e523a9b5-b54d-4e38-a07c-8fff05693ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:42.020581 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523a9b5-b54d-4e38-a07c-8fff05693ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:42.021067 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523a9b5-b54d-4e38-a07c-8fff05693ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:42.021564 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523a9b5-b54d-4e38-a07c-8fff05693ce4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:42.030133 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e523a9b5-b54d-4e38-a07c-8fff05693ce4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:42.031841 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e523a9b5-b54d-4e38-a07c-8fff05693ce4 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:42.043550 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-e523a9b5-b54d-4e38-a07c-8fff05693ce4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 04 00:48:42.043550 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:42.043550 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:42.043550 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:42.043550 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:42.043550 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:42.043550 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:42.043550 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:42.043550 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:42.043550 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:42.043550 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:42.043550 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:42.043550 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:48:42.043550 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:48:42.043550 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:48:42.043550 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:48:42.044692 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:42.044692 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:42.044692 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:48:42.044692 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:42.044692 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:48:42.044692 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:42.044692 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:42.044692 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:42.044692 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:42.044692 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:42.044692 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:42.044692 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:42.044692 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:42.044692 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:42.044692 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:42.044692 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:42.044692 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:42.044692 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:42.045612 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:42.045612 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:42.045612 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:48:42.045612 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:48:42.045612 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:48:42.045612 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:48:42.045612 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:48:42.045612 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:48:42.045612 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 04 00:48:42.045612 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:42.049703 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 337/679] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:48:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:42.049985 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb513c-5a59-4149-8373-a4d9e2b868c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:42.050485 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e523a9b5-b54d-4e38-a07c-8fff05693ce4 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:42.050613 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb513c-5a59-4149-8373-a4d9e2b868c7 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:42.051065 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb513c-5a59-4149-8373-a4d9e2b868c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:42.051186 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb513c-5a59-4149-8373-a4d9e2b868c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:42.057936 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb513c-5a59-4149-8373-a4d9e2b868c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:42.059627 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb513c-5a59-4149-8373-a4d9e2b868c7 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:42.071348 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-67eb513c-5a59-4149-8373-a4d9e2b868c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:42.071348 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:42.071348 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:42.071348 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:42.071348 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:42.071348 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:42.071348 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:42.071348 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:42.071348 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:42.071348 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:42.071348 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:42.071348 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:42.071348 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:48:42.071348 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:48:42.071348 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:48:42.072226 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:48:42.072226 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:42.072226 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:42.072226 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:42.072226 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:42.072226 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:42.072226 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:42.072226 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:42.072226 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:42.072226 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:42.072226 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:42.072226 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:42.072226 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:42.072226 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:42.072226 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:42.072226 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:42.072226 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:42.077785 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:48:42.077785 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:48:42.077785 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:48:42.077785 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:48:42.077785 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:48:42.077785 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:48:42.077785 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:42.077785 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:42.077785 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 343/680] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:48:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:42.077785 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab344260-2484-4744-b233-fb3018983666 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:42.077785 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab344260-2484-4744-b233-fb3018983666 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:42.078389 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab344260-2484-4744-b233-fb3018983666 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:42.078389 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 344/681] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:42.078389 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab344260-2484-4744-b233-fb3018983666 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:42.104338 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:42.161047 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e523a9b5-b54d-4e38-a07c-8fff05693ce4 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:42.195553 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d91eedb-bf9d-42dd-81d3-36af64967de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:42.196151 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91eedb-bf9d-42dd-81d3-36af64967de3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:42.196441 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91eedb-bf9d-42dd-81d3-36af64967de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:42.196728 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91eedb-bf9d-42dd-81d3-36af64967de3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:42.197444 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d91eedb-bf9d-42dd-81d3-36af64967de3 None admin] RBAC: Adding query filter params (role.id=6e3e398ba29e436e8532a45d2fdf1321, scope.project.id=5100a8d316d243fb8f82e9044e1b3b06, user.id=861c278f8bf34b3bb407b2e051119ee9) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:42.197700 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab344260-2484-4744-b233-fb3018983666 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:42.197821 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d91eedb-bf9d-42dd-81d3-36af64967de3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:42.199145 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d91eedb-bf9d-42dd-81d3-36af64967de3 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:42.205945 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 338/682] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:48:42 2022] GET /identity/v3/role_assignments?user.id=861c278f8bf34b3bb407b2e051119ee9&scope.project.id=5100a8d316d243fb8f82e9044e1b3b06&role.id=6e3e398ba29e436e8532a45d2fdf1321 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:42.206733 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d91eedb-bf9d-42dd-81d3-36af64967de3 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:42.236691 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:42.258744 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d83223-a6ee-49e3-8875-ca9c45836944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:42.259771 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d83223-a6ee-49e3-8875-ca9c45836944 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:42.260212 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d83223-a6ee-49e3-8875-ca9c45836944 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:42.260667 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d83223-a6ee-49e3-8875-ca9c45836944 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:42.261770 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d83223-a6ee-49e3-8875-ca9c45836944 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:42.262266 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d83223-a6ee-49e3-8875-ca9c45836944 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:42.265077 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d83223-a6ee-49e3-8875-ca9c45836944 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:42.291853 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 345/683] 10.222.0.27 () {56 vars in 1135 bytes} [Sun Sep 4 00:48:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:42.294336 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e603c66-cde8-48d4-8a31-023a9dac8be8 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:42.294794 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e603c66-cde8-48d4-8a31-023a9dac8be8 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:42.295231 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e603c66-cde8-48d4-8a31-023a9dac8be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:42.321620 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-2e603c66-cde8-48d4-8a31-023a9dac8be8 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:42.323327 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2e603c66-cde8-48d4-8a31-023a9dac8be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:42.359710 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d91eedb-bf9d-42dd-81d3-36af64967de3 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:42.418346 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b83cc7-4adc-49bc-b33f-8220ef3c06e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:42.418965 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b83cc7-4adc-49bc-b33f-8220ef3c06e8 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:42.419245 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b83cc7-4adc-49bc-b33f-8220ef3c06e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:42.419517 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b83cc7-4adc-49bc-b33f-8220ef3c06e8 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:42.420211 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b83cc7-4adc-49bc-b33f-8220ef3c06e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:42.420541 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b83cc7-4adc-49bc-b33f-8220ef3c06e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:42.421960 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b83cc7-4adc-49bc-b33f-8220ef3c06e8 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:42.423070 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 346/684] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:42.424644 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e603c66-cde8-48d4-8a31-023a9dac8be8 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:42.442269 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 339/685] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:48:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:42.443295 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b83cc7-4adc-49bc-b33f-8220ef3c06e8 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:42.445561 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:42.533493 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:42.559679 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b83cc7-4adc-49bc-b33f-8220ef3c06e8 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:42.571305 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e603c66-cde8-48d4-8a31-023a9dac8be8 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:42.613762 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfcc4ff-4a25-45f8-b808-d878da0eecb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:42.613952 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab3d896-b5d5-43c4-82c3-2874e95ef967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:42.614626 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab3d896-b5d5-43c4-82c3-2874e95ef967 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:42.614909 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab3d896-b5d5-43c4-82c3-2874e95ef967 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:42.615087 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfcc4ff-4a25-45f8-b808-d878da0eecb7 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:42.615203 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab3d896-b5d5-43c4-82c3-2874e95ef967 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:42.615616 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfcc4ff-4a25-45f8-b808-d878da0eecb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:42.615885 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab3d896-b5d5-43c4-82c3-2874e95ef967 None admin] RBAC: Adding query filter params (role.id=6e3e398ba29e436e8532a45d2fdf1321, scope.project.id=5100a8d316d243fb8f82e9044e1b3b06, user.id=6297dbc521e94330ae9294b942144181) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:42.616190 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab3d896-b5d5-43c4-82c3-2874e95ef967 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:42.616296 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfcc4ff-4a25-45f8-b808-d878da0eecb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:42.617734 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab3d896-b5d5-43c4-82c3-2874e95ef967 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:42.624843 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 340/686] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:48:42 2022] GET /identity/v3/role_assignments?user.id=6297dbc521e94330ae9294b942144181&scope.project.id=5100a8d316d243fb8f82e9044e1b3b06&role.id=6e3e398ba29e436e8532a45d2fdf1321 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:48:42.719891 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfcc4ff-4a25-45f8-b808-d878da0eecb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:42.721658 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfcc4ff-4a25-45f8-b808-d878da0eecb7 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:42.744840 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:42.765992 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab3d896-b5d5-43c4-82c3-2874e95ef967 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:42.859059 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 347/687] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:48:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:48:42.860398 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de729f46-cca3-4e4c-9660-f9c8c50179ba None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:42.860717 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de729f46-cca3-4e4c-9660-f9c8c50179ba None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:42.861057 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de729f46-cca3-4e4c-9660-f9c8c50179ba None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:42.862729 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 348/688] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:42.867860 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de729f46-cca3-4e4c-9660-f9c8c50179ba None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:42.891914 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab3d896-b5d5-43c4-82c3-2874e95ef967 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:42.932936 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c41da-1ca7-4eb0-9891-9ebc06121c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:42.933725 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c41da-1ca7-4eb0-9891-9ebc06121c9c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:42.934078 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c41da-1ca7-4eb0-9891-9ebc06121c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:42.934439 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c41da-1ca7-4eb0-9891-9ebc06121c9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:42.943590 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:42.947846 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c41da-1ca7-4eb0-9891-9ebc06121c9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:42.949530 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c41da-1ca7-4eb0-9891-9ebc06121c9c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:42.966377 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-254c41da-1ca7-4eb0-9891-9ebc06121c9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:42.966377 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:42.966377 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:42.966377 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:42.966377 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:42.966377 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:42.966377 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:42.966377 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:42.966377 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:42.966377 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:42.966377 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:42.966377 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:42.966377 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:42.966377 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:42.966377 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:42.966377 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:42.967334 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:42.967334 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:42.967334 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:42.967334 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:42.967334 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:42.967334 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:42.967334 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:42.967334 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:42.967334 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:42.967334 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:42.967334 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:42.967334 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:42.967334 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:42.967334 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:42.967334 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:42.967334 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:42.967334 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:42.968250 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:42.968250 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:42.968250 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:42.968250 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 341/689] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:42.969372 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d2282-6776-408b-81b1-f4299eab05d5 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:42.969372 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d2282-6776-408b-81b1-f4299eab05d5 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:42.969538 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d2282-6776-408b-81b1-f4299eab05d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:42.991171 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-fe8d2282-6776-408b-81b1-f4299eab05d5 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:42.993374 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-fe8d2282-6776-408b-81b1-f4299eab05d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:43.002093 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de729f46-cca3-4e4c-9660-f9c8c50179ba None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:43.016677 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:43.045613 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db31d0b-5f92-4c63-a7c8-e728c8e2fa06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:43.046580 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db31d0b-5f92-4c63-a7c8-e728c8e2fa06 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:43.047115 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db31d0b-5f92-4c63-a7c8-e728c8e2fa06 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:43.047604 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db31d0b-5f92-4c63-a7c8-e728c8e2fa06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:43.061763 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db31d0b-5f92-4c63-a7c8-e728c8e2fa06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:43.064415 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db31d0b-5f92-4c63-a7c8-e728c8e2fa06 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:43.078310 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 342/690] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:43.082886 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d2282-6776-408b-81b1-f4299eab05d5 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:43.087634 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-6db31d0b-5f92-4c63-a7c8-e728c8e2fa06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:43.087634 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:43.087634 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:43.087634 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:43.087634 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:43.087634 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:43.087634 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:43.087634 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:43.087634 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:43.087634 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:43.087634 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:43.087634 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:43.087634 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:43.087634 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:43.087634 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:43.087634 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:43.088553 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:43.088553 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:43.088553 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:43.088553 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:43.088553 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:43.088553 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:43.088553 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:43.088553 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:43.088553 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:43.088553 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:43.088553 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:43.088553 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:43.088553 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:43.088553 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:43.088553 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:43.088553 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:43.088553 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:43.089452 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:43.089452 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:43.089452 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:43.091617 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 349/691] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:43.097657 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db31d0b-5f92-4c63-a7c8-e728c8e2fa06 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:43.260484 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db31d0b-5f92-4c63-a7c8-e728c8e2fa06 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:43.293728 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc64c1-721f-42e7-ae5d-db6484188bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:43.294404 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc64c1-721f-42e7-ae5d-db6484188bcf None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:43.294693 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc64c1-721f-42e7-ae5d-db6484188bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:43.294996 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc64c1-721f-42e7-ae5d-db6484188bcf None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:43.295652 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc64c1-721f-42e7-ae5d-db6484188bcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:43.295964 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc64c1-721f-42e7-ae5d-db6484188bcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:43.301232 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc64c1-721f-42e7-ae5d-db6484188bcf None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:43.314524 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d2282-6776-408b-81b1-f4299eab05d5 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:43.326148 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 350/692] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:43.339197 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc64c1-721f-42e7-ae5d-db6484188bcf None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:43.339916 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:43.355772 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c09b6-b285-4e23-95f1-a7e3a6176922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:43.365722 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c09b6-b285-4e23-95f1-a7e3a6176922 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:43.365722 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c09b6-b285-4e23-95f1-a7e3a6176922 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:43.365722 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c09b6-b285-4e23-95f1-a7e3a6176922 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:43.365722 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615c09b6-b285-4e23-95f1-a7e3a6176922 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:43.365722 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615c09b6-b285-4e23-95f1-a7e3a6176922 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:43.365722 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615c09b6-b285-4e23-95f1-a7e3a6176922 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:43.373118 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 343/693] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:43.385490 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c09b6-b285-4e23-95f1-a7e3a6176922 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:43.394776 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:43.489821 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc64c1-721f-42e7-ae5d-db6484188bcf None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:43.534924 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c50b27-3a49-4e25-be8a-e5b577743a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:43.535927 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c50b27-3a49-4e25-be8a-e5b577743a51 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:43.536415 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c50b27-3a49-4e25-be8a-e5b577743a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:43.536928 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c50b27-3a49-4e25-be8a-e5b577743a51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:43.542651 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c09b6-b285-4e23-95f1-a7e3a6176922 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:43.544698 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c50b27-3a49-4e25-be8a-e5b577743a51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:43.546396 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c50b27-3a49-4e25-be8a-e5b577743a51 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:43.559569 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-80c50b27-3a49-4e25-be8a-e5b577743a51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:43.559569 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:43.559569 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:43.559569 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:43.559569 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:43.559569 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:43.559569 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:43.559569 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:43.559569 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:43.559569 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:43.559569 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:43.559569 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:43.559569 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:48:43.559569 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:48:43.559569 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:48:43.560633 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:48:43.560633 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:43.560633 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:43.560633 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:43.560633 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:43.560633 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:43.560633 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:43.560633 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:43.560633 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:43.560633 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:43.560633 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:43.560633 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:43.560633 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:43.560633 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:43.560633 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:43.560633 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:43.560633 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:43.561583 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:48:43.561583 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:48:43.561583 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:48:43.561583 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:48:43.561583 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:48:43.561583 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:48:43.561583 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:43.561583 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:43.564123 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 351/694] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:48:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:48:43.565436 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c50b27-3a49-4e25-be8a-e5b577743a51 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:43.587747 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ab134-e5ca-4bb8-9c0f-d7a2831dda2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:43.588395 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ab134-e5ca-4bb8-9c0f-d7a2831dda2e None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:43.588738 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ab134-e5ca-4bb8-9c0f-d7a2831dda2e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:43.589160 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ab134-e5ca-4bb8-9c0f-d7a2831dda2e None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:43.589809 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ab134-e5ca-4bb8-9c0f-d7a2831dda2e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:43.591397 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ab134-e5ca-4bb8-9c0f-d7a2831dda2e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:43.655567 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:43.670841 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c50b27-3a49-4e25-be8a-e5b577743a51 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:43.707787 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a4dc6-e529-4707-a524-126fa1dffc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:43.707787 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a4dc6-e529-4707-a524-126fa1dffc86 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:43.707787 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a4dc6-e529-4707-a524-126fa1dffc86 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:43.707787 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a4dc6-e529-4707-a524-126fa1dffc86 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:43.709531 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 344/695] 10.222.0.27 () {60 vars in 1132 bytes} [Sun Sep 4 00:48:43 2022] POST /identity/v3/users => generated 254 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:48:43.711939 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ab134-e5ca-4bb8-9c0f-d7a2831dda2e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:43.725144 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5a4dc6-e529-4707-a524-126fa1dffc86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:43.734097 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-2b5a4dc6-e529-4707-a524-126fa1dffc86 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:48:43.734097 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:43.734097 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:43.734097 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:43.734097 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:43.734097 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:43.734097 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:43.734097 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:43.734097 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:43.734097 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:43.734097 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:43.734097 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:43.734097 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:43.734097 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:43.734097 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:43.734097 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:43.735263 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:43.735263 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:43.735263 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:43.735263 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:43.735263 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:43.735263 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:43.735263 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:43.735263 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:43.735263 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:43.735263 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:43.735263 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:43.735263 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:43.735263 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:43.735263 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:43.735263 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:43.735263 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:43.736114 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:43.736114 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:43.736114 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:43.736114 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:43.736114 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:43.736114 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:43.736114 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:43.736114 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:43.736114 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:43.736114 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:43.736114 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:48:43.736114 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:43.741796 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 352/696] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:48:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:43.742718 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a4dc6-e529-4707-a524-126fa1dffc86 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:43.781366 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:43.858124 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ab134-e5ca-4bb8-9c0f-d7a2831dda2e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:43.902722 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484eedb-d850-48c5-84c2-7151339ae082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:43.903713 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484eedb-d850-48c5-84c2-7151339ae082 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:43.904307 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484eedb-d850-48c5-84c2-7151339ae082 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:43.904902 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484eedb-d850-48c5-84c2-7151339ae082 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:43.905861 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484eedb-d850-48c5-84c2-7151339ae082 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:43.906671 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484eedb-d850-48c5-84c2-7151339ae082 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:43.908646 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484eedb-d850-48c5-84c2-7151339ae082 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:43.923815 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 345/697] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:48:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:43.930341 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484eedb-d850-48c5-84c2-7151339ae082 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:43.955849 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a4dc6-e529-4707-a524-126fa1dffc86 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:44.004703 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e9da3c-7606-4bd9-8e3d-1732e0e2d671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:44.005835 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e9da3c-7606-4bd9-8e3d-1732e0e2d671 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:44.006127 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e9da3c-7606-4bd9-8e3d-1732e0e2d671 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:44.006409 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e9da3c-7606-4bd9-8e3d-1732e0e2d671 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:44.007239 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e9da3c-7606-4bd9-8e3d-1732e0e2d671 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:44.007551 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e9da3c-7606-4bd9-8e3d-1732e0e2d671 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:44.008993 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e9da3c-7606-4bd9-8e3d-1732e0e2d671 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:44.021097 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 353/698] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:48:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:44.027280 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e9da3c-7606-4bd9-8e3d-1732e0e2d671 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:44.037190 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:44.054001 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:44.061804 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484eedb-d850-48c5-84c2-7151339ae082 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:44.104270 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4c292-fd2c-4db4-a28e-72d630169767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:44.105643 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4c292-fd2c-4db4-a28e-72d630169767 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:44.105925 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4c292-fd2c-4db4-a28e-72d630169767 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:44.106198 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4c292-fd2c-4db4-a28e-72d630169767 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:44.106927 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c4c292-fd2c-4db4-a28e-72d630169767 None admin] RBAC: Adding query filter params (role.id=6e3e398ba29e436e8532a45d2fdf1321, scope.project.id=5100a8d316d243fb8f82e9044e1b3b06, user.id=2297d8b3fe8e47eb9fd39a6e3eea8f54) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:44.107232 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c4c292-fd2c-4db4-a28e-72d630169767 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:44.108742 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c4c292-fd2c-4db4-a28e-72d630169767 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:44.116343 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 346/699] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:48:43 2022] GET /identity/v3/role_assignments?user.id=2297d8b3fe8e47eb9fd39a6e3eea8f54&scope.project.id=5100a8d316d243fb8f82e9044e1b3b06&role.id=6e3e398ba29e436e8532a45d2fdf1321 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:44.155224 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e9da3c-7606-4bd9-8e3d-1732e0e2d671 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:44.195113 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a35a4c-40bd-467c-b414-bbb45094a56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:44.195763 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a35a4c-40bd-467c-b414-bbb45094a56c None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:44.196041 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a35a4c-40bd-467c-b414-bbb45094a56c None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:44.196312 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a35a4c-40bd-467c-b414-bbb45094a56c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:44.209475 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a35a4c-40bd-467c-b414-bbb45094a56c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:44.210967 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a35a4c-40bd-467c-b414-bbb45094a56c None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:44.226502 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-89a35a4c-40bd-467c-b414-bbb45094a56c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:44.226502 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:44.226502 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:44.226502 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:44.226502 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:44.226502 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:44.226502 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:44.226502 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:44.226502 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:44.226502 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:44.226502 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:44.226502 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:44.226502 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:44.226502 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:44.226502 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:44.226502 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:44.227513 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:44.227513 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:44.227513 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:44.227513 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:44.227513 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:44.227513 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:44.227513 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:44.227513 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:44.227513 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:44.227513 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:44.227513 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:44.227513 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:44.227513 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:44.227513 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:44.227513 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:44.227513 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:44.227513 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:44.228392 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:44.228392 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:44.228392 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:44.228392 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 354/700] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:48:44.233591 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4c292-fd2c-4db4-a28e-72d630169767 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:44.489672 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04017a0-b32f-4bb4-b653-194eb13bc0cb None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:44.489859 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04017a0-b32f-4bb4-b653-194eb13bc0cb None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:44.490359 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04017a0-b32f-4bb4-b653-194eb13bc0cb None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:44.491694 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 355/701] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 00:48:44.498291 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9dd0c9-d3a7-4151-af66-89c71ef4e1c1 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:44.498579 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9dd0c9-d3a7-4151-af66-89c71ef4e1c1 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:44.498856 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9dd0c9-d3a7-4151-af66-89c71ef4e1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:44.500179 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4c292-fd2c-4db4-a28e-72d630169767 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:44.531633 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-8a9dd0c9-d3a7-4151-af66-89c71ef4e1c1 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:44.533072 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8a9dd0c9-d3a7-4151-af66-89c71ef4e1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:44.548526 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2fba10-f2a9-4e0b-998f-ce4e65c129ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:44.548526 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2fba10-f2a9-4e0b-998f-ce4e65c129ef None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:44.548526 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2fba10-f2a9-4e0b-998f-ce4e65c129ef None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:44.548526 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2fba10-f2a9-4e0b-998f-ce4e65c129ef None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:44.549013 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2fba10-f2a9-4e0b-998f-ce4e65c129ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:44.549013 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2fba10-f2a9-4e0b-998f-ce4e65c129ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:44.549360 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2fba10-f2a9-4e0b-998f-ce4e65c129ef None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:44.563258 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 347/702] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:44.569268 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2fba10-f2a9-4e0b-998f-ce4e65c129ef None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:44.600639 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:44.615377 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:44.635910 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 356/703] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:44.682198 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2fba10-f2a9-4e0b-998f-ce4e65c129ef None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:44.717913 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e52ae-d629-464d-98b0-a74e28db775d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:44.718541 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e52ae-d629-464d-98b0-a74e28db775d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:44.718821 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e52ae-d629-464d-98b0-a74e28db775d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:44.719093 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e52ae-d629-464d-98b0-a74e28db775d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:44.726901 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e52ae-d629-464d-98b0-a74e28db775d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:44.728334 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e52ae-d629-464d-98b0-a74e28db775d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:44.740537 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-da5e52ae-d629-464d-98b0-a74e28db775d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 04 00:48:44.740537 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:44.740537 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:44.740537 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:44.740537 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:44.740537 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:44.740537 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:44.740537 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:44.740537 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:44.740537 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:44.740537 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:44.740537 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:44.740537 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:48:44.740537 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:48:44.740537 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:48:44.740537 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:48:44.741607 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:44.741607 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:44.741607 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:48:44.741607 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:44.741607 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:48:44.741607 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:44.741607 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:44.741607 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:44.741607 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:44.741607 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:44.741607 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:44.741607 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:44.741607 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:44.741607 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:44.741607 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:44.741607 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:44.741607 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:44.741607 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:44.742532 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:44.742532 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:44.742532 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:48:44.742532 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:48:44.742532 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:48:44.742532 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:48:44.742532 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:48:44.742532 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:48:44.742532 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 04 00:48:44.742532 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:44.742532 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 348/704] 10.222.0.27 () {56 vars in 1093 bytes} [Sun Sep 4 00:48:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:48:44.749515 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9dd0c9-d3a7-4151-af66-89c71ef4e1c1 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:44.917554 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9dd0c9-d3a7-4151-af66-89c71ef4e1c1 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:44.964260 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f347a-82fd-4bdc-a20f-6c35349471ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:44.964901 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f347a-82fd-4bdc-a20f-6c35349471ea None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:44.965180 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f347a-82fd-4bdc-a20f-6c35349471ea None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:44.965450 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f347a-82fd-4bdc-a20f-6c35349471ea None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:44.966180 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3f347a-82fd-4bdc-a20f-6c35349471ea None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:44.966486 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3f347a-82fd-4bdc-a20f-6c35349471ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:44.968003 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3f347a-82fd-4bdc-a20f-6c35349471ea None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:44.999909 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 357/705] 10.222.0.27 () {56 vars in 1133 bytes} [Sun Sep 4 00:48:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:45.006623 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e52ae-d629-464d-98b0-a74e28db775d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:45.007917 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f347a-82fd-4bdc-a20f-6c35349471ea None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:45.056999 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:45.115810 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f347a-82fd-4bdc-a20f-6c35349471ea None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:45.144845 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:45.156796 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e52ae-d629-464d-98b0-a74e28db775d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:45.166321 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c514aea-83f8-45e4-a115-615703f6b16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:45.166967 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c514aea-83f8-45e4-a115-615703f6b16c None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:45.167312 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c514aea-83f8-45e4-a115-615703f6b16c None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:45.167591 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c514aea-83f8-45e4-a115-615703f6b16c None admin] PATH_INFO: `/v3/services/network` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:45.168182 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c514aea-83f8-45e4-a115-615703f6b16c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:45.170280 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c514aea-83f8-45e4-a115-615703f6b16c None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:45.184332 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-0c514aea-83f8-45e4-a115-615703f6b16c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 04 00:48:45.184332 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:45.184332 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 00:48:45.184332 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 00:48:45.184332 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 00:48:45.184332 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 00:48:45.184332 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 00:48:45.184332 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:48:45.184332 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 04 00:48:45.184332 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:45.184332 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 00:48:45.184332 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:45.184332 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:45.184332 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:45.184332 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:45.184332 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:45.184332 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:45.185567 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:45.185567 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:45.185567 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:45.185567 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:45.185567 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:45.185567 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:45.185567 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 00:48:45.185567 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 00:48:45.185567 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 00:48:45.185567 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 00:48:45.185567 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:45.185567 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:45.185567 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:45.185567 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:45.185567 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:45.185567 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:45.186451 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:45.186451 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:45.186451 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:45.186451 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:45.186451 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:45.186451 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:45.186451 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:45.186451 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:45.186451 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:45.186451 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:45.186451 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 00:48:45.186451 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:48:45.186451 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 04 00:48:45.186451 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:45.186451 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 358/706] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:48:45 2022] GET /identity/v3/services/network => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:48:45.187252 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba78bd-6374-4df2-8cd4-2aebd02acd12 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:45.187252 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba78bd-6374-4df2-8cd4-2aebd02acd12 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:45.193065 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba78bd-6374-4df2-8cd4-2aebd02acd12 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:45.193198 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 359/707] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:45.193913 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba78bd-6374-4df2-8cd4-2aebd02acd12 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:45.210818 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3edc732-c063-40aa-8b33-ce29325b4ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:45.212054 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3edc732-c063-40aa-8b33-ce29325b4ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:45.212639 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3edc732-c063-40aa-8b33-ce29325b4ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:45.213334 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3edc732-c063-40aa-8b33-ce29325b4ec6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:45.297061 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3edc732-c063-40aa-8b33-ce29325b4ec6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:45.298875 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3edc732-c063-40aa-8b33-ce29325b4ec6 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:45.312143 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba78bd-6374-4df2-8cd4-2aebd02acd12 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:45.324281 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:45.350230 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a5522-628d-469e-9552-abf3c42b2be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:45.350830 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a5522-628d-469e-9552-abf3c42b2be3 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:45.351220 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a5522-628d-469e-9552-abf3c42b2be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:45.351695 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a5522-628d-469e-9552-abf3c42b2be3 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:45.352484 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a5522-628d-469e-9552-abf3c42b2be3 None admin] RBAC: Adding query filter params (name=network) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:45.353130 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a5522-628d-469e-9552-abf3c42b2be3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:45.355121 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a5522-628d-469e-9552-abf3c42b2be3 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:45.364545 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 360/708] 10.222.0.27 () {56 vars in 1112 bytes} [Sun Sep 4 00:48:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:45.365870 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc644bc4-d40c-4d08-ad56-e965dba54979 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:45.366266 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc644bc4-d40c-4d08-ad56-e965dba54979 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:45.366660 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc644bc4-d40c-4d08-ad56-e965dba54979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:45.385828 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-dc644bc4-d40c-4d08-ad56-e965dba54979 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:45.387165 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-dc644bc4-d40c-4d08-ad56-e965dba54979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:45.389050 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:45.412795 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 349/709] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:48:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:48:45.413873 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3edc732-c063-40aa-8b33-ce29325b4ec6 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:45.463984 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 361/710] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:45.473525 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc644bc4-d40c-4d08-ad56-e965dba54979 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:45.524568 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3edc732-c063-40aa-8b33-ce29325b4ec6 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:45.570846 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db28f0-8491-473f-92ba-2a518d98a4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:45.572422 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db28f0-8491-473f-92ba-2a518d98a4e3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:45.573466 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db28f0-8491-473f-92ba-2a518d98a4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:45.574049 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db28f0-8491-473f-92ba-2a518d98a4e3 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:45.575958 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db28f0-8491-473f-92ba-2a518d98a4e3 None admin] RBAC: Adding query filter params (type=network) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:45.577003 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db28f0-8491-473f-92ba-2a518d98a4e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:45.578514 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db28f0-8491-473f-92ba-2a518d98a4e3 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:45.590403 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 350/711] 10.222.0.27 () {56 vars in 1112 bytes} [Sun Sep 4 00:48:45 2022] GET /identity/v3/services?type=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:45.625475 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc644bc4-d40c-4d08-ad56-e965dba54979 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:45.628895 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:45.679273 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5f5eb-89ed-4bda-bd6f-289a3774a5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:45.680165 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5f5eb-89ed-4bda-bd6f-289a3774a5b0 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:45.680618 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5f5eb-89ed-4bda-bd6f-289a3774a5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:45.681026 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5f5eb-89ed-4bda-bd6f-289a3774a5b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:45.696897 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a5f5eb-89ed-4bda-bd6f-289a3774a5b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:45.698643 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a5f5eb-89ed-4bda-bd6f-289a3774a5b0 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:45.718020 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-42a5f5eb-89ed-4bda-bd6f-289a3774a5b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:45.718020 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:45.718020 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:45.718020 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:45.718020 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:45.718020 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:45.718020 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:45.718020 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:45.718020 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:45.718020 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:45.718020 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:45.718020 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:45.718020 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:45.718020 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:45.718020 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:45.718020 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:45.719231 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:45.719231 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:45.719231 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:45.719231 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:45.719231 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:45.719231 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:45.719231 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:45.719231 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:45.719231 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:45.719231 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:45.719231 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:45.719231 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:45.719231 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:45.719231 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:45.719231 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:45.719231 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:45.719231 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:45.721095 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:45.721095 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:45.721095 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:45.722494 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 362/712] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:45.728554 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db28f0-8491-473f-92ba-2a518d98a4e3 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:45.785211 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:45.847324 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db28f0-8491-473f-92ba-2a518d98a4e3 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:45.849593 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5f5eb-89ed-4bda-bd6f-289a3774a5b0 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:45.887508 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48331e6-bc79-41b7-9251-5679cf61915d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:45.888486 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48331e6-bc79-41b7-9251-5679cf61915d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:45.888901 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48331e6-bc79-41b7-9251-5679cf61915d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:45.889293 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48331e6-bc79-41b7-9251-5679cf61915d None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:45.890181 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48331e6-bc79-41b7-9251-5679cf61915d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:45.890637 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48331e6-bc79-41b7-9251-5679cf61915d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:45.892745 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48331e6-bc79-41b7-9251-5679cf61915d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:45.906560 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 351/713] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:45.914201 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48331e6-bc79-41b7-9251-5679cf61915d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:45.967719 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5f5eb-89ed-4bda-bd6f-289a3774a5b0 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:46.034120 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb1716-2282-4d65-a93b-ebcb46668adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:46.034774 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb1716-2282-4d65-a93b-ebcb46668adf None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:46.035057 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb1716-2282-4d65-a93b-ebcb46668adf None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:46.035544 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb1716-2282-4d65-a93b-ebcb46668adf None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:46.035913 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cb1716-2282-4d65-a93b-ebcb46668adf None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:46.037326 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cb1716-2282-4d65-a93b-ebcb46668adf None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:46.069838 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-b8cb1716-2282-4d65-a93b-ebcb46668adf None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 04 00:48:46.069838 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:46.069838 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 00:48:46.069838 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 00:48:46.069838 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 00:48:46.069838 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 00:48:46.069838 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 00:48:46.069838 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:48:46.069838 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 04 00:48:46.069838 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:46.069838 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 00:48:46.069838 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:46.069838 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:46.069838 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:46.069838 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:46.069838 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:46.069838 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:46.071068 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:46.071068 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:46.071068 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:46.071068 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:46.071068 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:46.071068 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:46.071068 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 00:48:46.071068 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 00:48:46.071068 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 00:48:46.071068 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 00:48:46.071068 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:46.071068 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:46.071068 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:46.071068 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:46.071068 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:46.071068 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:46.072035 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:46.072035 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:46.072035 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:46.072035 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:46.072035 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:46.072035 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:46.072035 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:46.072035 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:46.072035 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:46.072035 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:46.072035 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 00:48:46.072035 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:48:46.072035 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 04 00:48:46.072035 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:46.072035 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 363/714] 10.222.0.27 () {56 vars in 1129 bytes} [Sun Sep 4 00:48:45 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:48:46.078055 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb1716-2282-4d65-a93b-ebcb46668adf None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:46.079249 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48331e6-bc79-41b7-9251-5679cf61915d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:46.088977 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:46.125010 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab76b0d-c561-4873-88a0-fe5a86c361a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:46.125010 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab76b0d-c561-4873-88a0-fe5a86c361a3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:46.125010 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab76b0d-c561-4873-88a0-fe5a86c361a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:46.125010 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab76b0d-c561-4873-88a0-fe5a86c361a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:46.129938 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab76b0d-c561-4873-88a0-fe5a86c361a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:46.131573 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab76b0d-c561-4873-88a0-fe5a86c361a3 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:46.144378 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-2ab76b0d-c561-4873-88a0-fe5a86c361a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:46.144378 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:46.144378 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:46.144378 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:46.144378 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:46.144378 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:46.144378 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:46.144378 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:46.144378 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:46.144378 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:46.144378 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:46.144378 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:46.144378 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:48:46.144378 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:48:46.144378 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:48:46.145499 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:48:46.145499 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:46.145499 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:46.145499 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:46.145499 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:46.145499 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:46.145499 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:46.145499 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:46.145499 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:46.145499 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:46.145499 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:46.145499 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:46.145499 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:46.145499 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:46.145499 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:46.145499 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:46.145499 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:46.146584 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:48:46.146584 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:48:46.146584 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:48:46.146584 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:48:46.146584 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:48:46.146584 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:48:46.146584 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:46.146584 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:46.147601 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:46.149381 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 352/715] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:48:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:46.155570 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab76b0d-c561-4873-88a0-fe5a86c361a3 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:46.253162 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb1716-2282-4d65-a93b-ebcb46668adf None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:46.290358 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab76b0d-c561-4873-88a0-fe5a86c361a3 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:46.293808 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9384450-edc0-456b-aa33-45757163b55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:46.294672 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9384450-edc0-456b-aa33-45757163b55a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:46.295169 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9384450-edc0-456b-aa33-45757163b55a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:46.295645 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9384450-edc0-456b-aa33-45757163b55a None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:46.296523 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9384450-edc0-456b-aa33-45757163b55a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:46.297047 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9384450-edc0-456b-aa33-45757163b55a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:46.298926 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9384450-edc0-456b-aa33-45757163b55a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:46.304294 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 364/716] 10.222.0.27 () {56 vars in 1124 bytes} [Sun Sep 4 00:48:46 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:46.306047 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5f750-5410-40fa-8a37-bce075025fb2 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:46.306418 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5f750-5410-40fa-8a37-bce075025fb2 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:46.306823 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5f750-5410-40fa-8a37-bce075025fb2 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:46.308334 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 365/717] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:46.309324 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d5f750-5410-40fa-8a37-bce075025fb2 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:46.330214 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5ef7f-805b-46e5-94d3-3edbfca5db54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:46.331098 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5ef7f-805b-46e5-94d3-3edbfca5db54 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:46.331539 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5ef7f-805b-46e5-94d3-3edbfca5db54 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:46.331943 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5ef7f-805b-46e5-94d3-3edbfca5db54 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:46.332811 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf5ef7f-805b-46e5-94d3-3edbfca5db54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:46.333306 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf5ef7f-805b-46e5-94d3-3edbfca5db54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:46.335807 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf5ef7f-805b-46e5-94d3-3edbfca5db54 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:46.353152 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 353/718] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:48:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:46.354615 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b632c39-2b9d-4f95-8ce7-a95fd3a506e5 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:46.355040 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b632c39-2b9d-4f95-8ce7-a95fd3a506e5 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:46.355452 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b632c39-2b9d-4f95-8ce7-a95fd3a506e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:46.379828 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1b632c39-2b9d-4f95-8ce7-a95fd3a506e5 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:46.381634 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1b632c39-2b9d-4f95-8ce7-a95fd3a506e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:46.403085 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:46.407460 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:46.429293 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d5f750-5410-40fa-8a37-bce075025fb2 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:46.461917 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 354/719] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:46.462833 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b632c39-2b9d-4f95-8ce7-a95fd3a506e5 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:46.475210 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb216f-845d-4fa8-a89d-c471f90c7fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:46.476501 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb216f-845d-4fa8-a89d-c471f90c7fef None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:46.477140 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb216f-845d-4fa8-a89d-c471f90c7fef None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:46.477708 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb216f-845d-4fa8-a89d-c471f90c7fef None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:46.478515 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbb216f-845d-4fa8-a89d-c471f90c7fef None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:46.479168 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbb216f-845d-4fa8-a89d-c471f90c7fef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:46.481374 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbb216f-845d-4fa8-a89d-c471f90c7fef None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:46.489071 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 366/720] 10.222.0.27 () {56 vars in 1124 bytes} [Sun Sep 4 00:48:46 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:46.581015 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b632c39-2b9d-4f95-8ce7-a95fd3a506e5 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:46.624168 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53328b09-7280-44a0-9eec-119d36ffcc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:46.624886 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53328b09-7280-44a0-9eec-119d36ffcc37 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:46.625164 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53328b09-7280-44a0-9eec-119d36ffcc37 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:46.625437 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53328b09-7280-44a0-9eec-119d36ffcc37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:46.626707 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53328b09-7280-44a0-9eec-119d36ffcc37 None admin] RBAC: Adding query filter params (role.id=c8a59d34ee2f4cefa6940cac778260c7, scope.project.id=5100a8d316d243fb8f82e9044e1b3b06, user.id=68bc00bda5194cd8a7928f8c4df4ccd1) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:46.627030 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53328b09-7280-44a0-9eec-119d36ffcc37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:46.628487 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53328b09-7280-44a0-9eec-119d36ffcc37 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:46.635237 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 355/721] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:48:46 2022] GET /identity/v3/role_assignments?user.id=68bc00bda5194cd8a7928f8c4df4ccd1&scope.project.id=5100a8d316d243fb8f82e9044e1b3b06&role.id=c8a59d34ee2f4cefa6940cac778260c7 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:46.794073 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee591347-54af-4bb7-8e9e-17dabd9410ea None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:46.795095 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee591347-54af-4bb7-8e9e-17dabd9410ea None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:46.795577 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee591347-54af-4bb7-8e9e-17dabd9410ea None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:46.797180 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 367/722] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:46.804093 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce0b29-915c-48e6-b84c-e396ba0e8126 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:46.804385 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce0b29-915c-48e6-b84c-e396ba0e8126 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:46.804743 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce0b29-915c-48e6-b84c-e396ba0e8126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:46.806465 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee591347-54af-4bb7-8e9e-17dabd9410ea None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:46.825088 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-66ce0b29-915c-48e6-b84c-e396ba0e8126 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:46.826266 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-66ce0b29-915c-48e6-b84c-e396ba0e8126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:46.845774 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:46.930010 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee591347-54af-4bb7-8e9e-17dabd9410ea None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:46.947057 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 356/723] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:46.982057 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de8374-733c-4144-9abc-184fa9e23e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:46.982700 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de8374-733c-4144-9abc-184fa9e23e96 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:46.982985 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de8374-733c-4144-9abc-184fa9e23e96 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:46.983265 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de8374-733c-4144-9abc-184fa9e23e96 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:46.999595 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de8374-733c-4144-9abc-184fa9e23e96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:47.006986 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-d4de8374-733c-4144-9abc-184fa9e23e96 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:47.006986 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:47.006986 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:47.006986 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:47.006986 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:47.006986 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:47.006986 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:47.006986 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:47.006986 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:47.006986 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:47.006986 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:47.006986 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:47.006986 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:47.006986 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:47.006986 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:47.006986 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:47.007907 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:47.007907 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:47.007907 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:47.007907 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:47.007907 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:47.007907 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:47.007907 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:47.007907 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:47.007907 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:47.007907 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:47.007907 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:47.007907 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:47.007907 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:47.007907 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:47.007907 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:47.007907 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:47.008870 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:47.008870 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:47.008870 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:47.008870 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:47.008870 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:47.008870 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:47.008870 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:47.008870 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:47.008870 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:47.008870 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:47.008870 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:47.008870 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:47.012918 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 368/724] 10.222.0.27 () {56 vars in 1102 bytes} [Sun Sep 4 00:48:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:47.018314 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce0b29-915c-48e6-b84c-e396ba0e8126 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:47.177549 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce0b29-915c-48e6-b84c-e396ba0e8126 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:47.231212 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f2363e-d4eb-40d5-9819-70760c82e127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:47.231843 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2363e-d4eb-40d5-9819-70760c82e127 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:47.232117 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2363e-d4eb-40d5-9819-70760c82e127 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:47.232380 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2363e-d4eb-40d5-9819-70760c82e127 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:47.233083 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f2363e-d4eb-40d5-9819-70760c82e127 None admin] RBAC: Adding query filter params (name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:47.233380 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f2363e-d4eb-40d5-9819-70760c82e127 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:47.234729 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f2363e-d4eb-40d5-9819-70760c82e127 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:47.247637 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 357/725] 10.222.0.27 () {56 vars in 1103 bytes} [Sun Sep 4 00:48:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:47.253105 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de8374-733c-4144-9abc-184fa9e23e96 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:47.276286 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:47.277640 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f2363e-d4eb-40d5-9819-70760c82e127 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:47.309956 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:47.420752 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f2363e-d4eb-40d5-9819-70760c82e127 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:47.457765 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de8374-733c-4144-9abc-184fa9e23e96 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:47.473017 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39f242-1efb-4080-80a2-ffe908c4e3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:47.473649 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39f242-1efb-4080-80a2-ffe908c4e3a6 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:47.473933 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39f242-1efb-4080-80a2-ffe908c4e3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:47.474496 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39f242-1efb-4080-80a2-ffe908c4e3a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:47.489591 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd39f242-1efb-4080-80a2-ffe908c4e3a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:47.490480 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-cd39f242-1efb-4080-80a2-ffe908c4e3a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:48:47.490480 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:47.490480 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:47.490480 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:47.490480 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:47.490480 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:47.490480 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:47.490480 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:47.490480 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:47.490480 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:47.490480 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:47.490480 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:47.490480 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:47.490480 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:47.490480 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:47.490480 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:47.491451 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:47.491451 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:47.491451 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:47.491451 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:47.491451 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:47.491451 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:47.491451 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:47.491451 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:47.491451 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:47.491451 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:47.491451 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:47.491451 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:47.491451 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:47.491451 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:47.491451 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:47.491451 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:47.492270 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:47.492270 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:47.492270 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:47.492270 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:47.492270 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:47.492270 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:47.492270 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:47.492270 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:47.492270 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:47.492270 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:47.492270 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:48:47.492270 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:47.492270 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 358/726] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:48:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:47.507063 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39f242-1efb-4080-80a2-ffe908c4e3a6 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:47.509229 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58634e5-c81a-45a3-8c29-01085fb820f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:47.510060 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58634e5-c81a-45a3-8c29-01085fb820f4 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:47.510532 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58634e5-c81a-45a3-8c29-01085fb820f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:47.510978 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58634e5-c81a-45a3-8c29-01085fb820f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:47.522521 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58634e5-c81a-45a3-8c29-01085fb820f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:47.524274 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58634e5-c81a-45a3-8c29-01085fb820f4 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:47.539984 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-f58634e5-c81a-45a3-8c29-01085fb820f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:47.539984 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:47.539984 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:47.539984 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:47.539984 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:47.539984 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:47.539984 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:47.539984 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:47.539984 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:47.539984 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:47.539984 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:47.539984 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:47.539984 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:47.539984 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:47.539984 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:47.539984 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:47.540961 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:47.540961 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:47.540961 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:47.540961 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:47.540961 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:47.540961 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:47.540961 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:47.540961 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:47.540961 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:47.540961 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:47.540961 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:47.540961 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:47.540961 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:47.540961 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:47.540961 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:47.540961 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:47.540961 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:47.541830 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:47.541830 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:47.541830 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:47.543853 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 369/727] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:47.550286 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58634e5-c81a-45a3-8c29-01085fb820f4 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:47.613026 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:47.656699 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58634e5-c81a-45a3-8c29-01085fb820f4 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:47.672814 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:47.694937 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49621ba7-4f57-4457-86a5-9ef6e38d9c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:47.695970 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49621ba7-4f57-4457-86a5-9ef6e38d9c1b None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:47.696505 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49621ba7-4f57-4457-86a5-9ef6e38d9c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:47.696979 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49621ba7-4f57-4457-86a5-9ef6e38d9c1b None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:47.697840 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49621ba7-4f57-4457-86a5-9ef6e38d9c1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:47.698346 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49621ba7-4f57-4457-86a5-9ef6e38d9c1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:47.699904 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49621ba7-4f57-4457-86a5-9ef6e38d9c1b None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:47.713553 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 370/728] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:47.716511 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39f242-1efb-4080-80a2-ffe908c4e3a6 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:47.719061 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49621ba7-4f57-4457-86a5-9ef6e38d9c1b None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:47.813179 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e4a5d-1200-4e74-946c-23064d2c188c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:47.814398 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e4a5d-1200-4e74-946c-23064d2c188c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:47.814920 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e4a5d-1200-4e74-946c-23064d2c188c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:47.815276 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e4a5d-1200-4e74-946c-23064d2c188c None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:47.815962 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10e4a5d-1200-4e74-946c-23064d2c188c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:47.816277 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10e4a5d-1200-4e74-946c-23064d2c188c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:47.817723 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10e4a5d-1200-4e74-946c-23064d2c188c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:47.825962 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49621ba7-4f57-4457-86a5-9ef6e38d9c1b None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:47.835979 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 359/729] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:48:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:47.849531 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e4a5d-1200-4e74-946c-23064d2c188c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:47.853493 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:47.869223 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e2856-84a7-44e8-ba91-a3710ffe008e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:47.869995 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e2856-84a7-44e8-ba91-a3710ffe008e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:47.870408 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e2856-84a7-44e8-ba91-a3710ffe008e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:47.870778 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e2856-84a7-44e8-ba91-a3710ffe008e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:47.879311 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3e2856-84a7-44e8-ba91-a3710ffe008e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:47.881202 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3e2856-84a7-44e8-ba91-a3710ffe008e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:47.894010 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-5f3e2856-84a7-44e8-ba91-a3710ffe008e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 04 00:48:47.894010 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:47.894010 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:47.894010 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:47.894010 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:47.894010 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:47.894010 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:47.894010 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:47.894010 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:47.894010 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:47.894010 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:47.894010 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:47.894010 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:48:47.894010 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:48:47.894010 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:48:47.894010 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:48:47.895142 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:47.895142 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:47.895142 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:48:47.895142 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:47.895142 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:48:47.895142 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:47.895142 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:47.895142 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:47.895142 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:47.895142 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:47.895142 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:47.895142 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:47.895142 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:47.895142 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:47.895142 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:47.895142 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:47.895142 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:47.895142 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:47.896287 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:47.896287 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:47.896287 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:48:47.896287 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:48:47.896287 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:48:47.896287 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:48:47.896287 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:48:47.896287 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:48:47.896287 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 04 00:48:47.896287 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:47.902674 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 371/730] 10.222.0.27 () {56 vars in 1117 bytes} [Sun Sep 4 00:48:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:48:47.903730 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e2856-84a7-44e8-ba91-a3710ffe008e None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:47.994817 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e4a5d-1200-4e74-946c-23064d2c188c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:48.030303 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:48.041343 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3803854-5381-4be5-bf56-7ff5cd2a5f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:48.041994 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3803854-5381-4be5-bf56-7ff5cd2a5f72 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:48.042276 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3803854-5381-4be5-bf56-7ff5cd2a5f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:48.042547 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3803854-5381-4be5-bf56-7ff5cd2a5f72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:48.056869 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3803854-5381-4be5-bf56-7ff5cd2a5f72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:48.058916 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3803854-5381-4be5-bf56-7ff5cd2a5f72 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:48.067749 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e2856-84a7-44e8-ba91-a3710ffe008e None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:48.077425 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-c3803854-5381-4be5-bf56-7ff5cd2a5f72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:48.077425 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:48.077425 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:48.077425 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:48.077425 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:48.077425 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:48.077425 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:48.077425 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:48.077425 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:48.077425 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:48.077425 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:48.077425 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:48.077425 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:48.077425 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:48.077425 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:48.077425 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:48.078391 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:48.078391 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:48.078391 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:48.078391 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:48.078391 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:48.078391 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:48.078391 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:48.078391 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:48.078391 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:48.078391 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:48.078391 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:48.078391 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:48.078391 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:48.078391 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:48.078391 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:48.078391 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:48.078391 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:48.079459 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:48.079459 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:48.079459 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:48.079459 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 360/731] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:48.087511 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f2751-c44d-4df0-bfd0-aead600b9316 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:48.087511 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f2751-c44d-4df0-bfd0-aead600b9316 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:48.087511 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f2751-c44d-4df0-bfd0-aead600b9316 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:48.087511 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 361/732] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:48.087511 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f2751-c44d-4df0-bfd0-aead600b9316 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:48.120781 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af978f1-5617-41a9-a666-5ca4e19f8b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:48.121812 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af978f1-5617-41a9-a666-5ca4e19f8b49 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:48.122319 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af978f1-5617-41a9-a666-5ca4e19f8b49 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:48.122893 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af978f1-5617-41a9-a666-5ca4e19f8b49 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:48.123922 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af978f1-5617-41a9-a666-5ca4e19f8b49 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:48.124429 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af978f1-5617-41a9-a666-5ca4e19f8b49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:48.126336 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af978f1-5617-41a9-a666-5ca4e19f8b49 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:48.145417 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 372/733] 10.222.0.27 () {56 vars in 1149 bytes} [Sun Sep 4 00:48:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:48.147136 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb967279-c9a1-4e65-ab15-f4f4b6c46938 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:48.147630 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb967279-c9a1-4e65-ab15-f4f4b6c46938 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:48.148142 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb967279-c9a1-4e65-ab15-f4f4b6c46938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:48.169073 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-fb967279-c9a1-4e65-ab15-f4f4b6c46938 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:48.170542 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-fb967279-c9a1-4e65-ab15-f4f4b6c46938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:48.202630 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:48.215265 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f2751-c44d-4df0-bfd0-aead600b9316 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:48.256502 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 373/734] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:48.256502 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb967279-c9a1-4e65-ab15-f4f4b6c46938 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:48.275082 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f3f617-8b57-47aa-88be-923e73450fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:48.276073 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f3f617-8b57-47aa-88be-923e73450fae None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:48.276600 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f3f617-8b57-47aa-88be-923e73450fae None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:48.277126 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f3f617-8b57-47aa-88be-923e73450fae None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:48.277986 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f3f617-8b57-47aa-88be-923e73450fae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:48.278490 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f3f617-8b57-47aa-88be-923e73450fae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:48.280284 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f3f617-8b57-47aa-88be-923e73450fae None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:48.306296 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 362/735] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:48:48.312299 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f3f617-8b57-47aa-88be-923e73450fae None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:48.361117 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb967279-c9a1-4e65-ab15-f4f4b6c46938 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:48.400021 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da4046-7d71-40bb-8fee-5e07d2b7af65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:48.400369 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da4046-7d71-40bb-8fee-5e07d2b7af65 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:48.400637 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da4046-7d71-40bb-8fee-5e07d2b7af65 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:48.400911 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da4046-7d71-40bb-8fee-5e07d2b7af65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:48.404433 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:48.461506 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:48.496538 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3da4046-7d71-40bb-8fee-5e07d2b7af65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:48.498120 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3da4046-7d71-40bb-8fee-5e07d2b7af65 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:48.515198 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f3f617-8b57-47aa-88be-923e73450fae None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:48.633681 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f07d9-6ef0-47a3-8dbc-72d5ad3dae60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:48.633681 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f07d9-6ef0-47a3-8dbc-72d5ad3dae60 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:48.633681 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f07d9-6ef0-47a3-8dbc-72d5ad3dae60 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:48.633681 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f07d9-6ef0-47a3-8dbc-72d5ad3dae60 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:48.634046 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327f07d9-6ef0-47a3-8dbc-72d5ad3dae60 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:48.634046 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327f07d9-6ef0-47a3-8dbc-72d5ad3dae60 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:48.643208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-327f07d9-6ef0-47a3-8dbc-72d5ad3dae60 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 04 00:48:48.643208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:48.643208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:48.643208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:48.643208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:48.643208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:48.643208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:48.643208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:48.643208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:48.643208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:48.643208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:48.643208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:48.643208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:48:48.643208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:48:48.643208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:48:48.643208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:48:48.644156 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:48.644156 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:48.644156 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:48:48.644156 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:48.644156 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:48:48.644156 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:48.644156 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:48.644156 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:48.644156 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:48.644156 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:48.644156 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:48.644156 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:48.644156 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:48.644156 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:48.644156 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:48.644156 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:48.644156 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:48.644156 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:48.645132 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:48.645132 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:48.645132 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:48:48.645132 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:48:48.645132 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:48:48.645132 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:48:48.645132 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:48:48.645132 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:48:48.645132 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 04 00:48:48.645132 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:48.645132 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 363/736] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:48:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:48:48.645736 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f07d9-6ef0-47a3-8dbc-72d5ad3dae60 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:48.680381 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 374/737] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:48:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:48:48.682308 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da4046-7d71-40bb-8fee-5e07d2b7af65 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:48.769125 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f07d9-6ef0-47a3-8dbc-72d5ad3dae60 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:48.772286 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:48.779957 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:48.810323 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9568ef1-691e-4eed-9ca7-a9577818f414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:48.811009 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9568ef1-691e-4eed-9ca7-a9577818f414 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:48.811652 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9568ef1-691e-4eed-9ca7-a9577818f414 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:48.811652 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9568ef1-691e-4eed-9ca7-a9577818f414 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:48.812188 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9568ef1-691e-4eed-9ca7-a9577818f414 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:48.813590 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9568ef1-691e-4eed-9ca7-a9577818f414 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:48.824830 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da4046-7d71-40bb-8fee-5e07d2b7af65 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:48.861889 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073b4cbd-e998-4196-a338-8fb1b3ff8207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:48.863214 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b4cbd-e998-4196-a338-8fb1b3ff8207 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:48.863647 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b4cbd-e998-4196-a338-8fb1b3ff8207 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:48.864069 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b4cbd-e998-4196-a338-8fb1b3ff8207 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:48.865011 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073b4cbd-e998-4196-a338-8fb1b3ff8207 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:48.865479 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073b4cbd-e998-4196-a338-8fb1b3ff8207 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:48.867075 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073b4cbd-e998-4196-a338-8fb1b3ff8207 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:48.890996 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 375/738] 10.222.0.27 () {56 vars in 1135 bytes} [Sun Sep 4 00:48:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:48.890996 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073b4cbd-e998-4196-a338-8fb1b3ff8207 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:48.920592 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 364/739] 10.222.0.27 () {60 vars in 1141 bytes} [Sun Sep 4 00:48:48 2022] POST /identity/v3/services => generated 240 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:48:48.924951 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9568ef1-691e-4eed-9ca7-a9577818f414 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:49.026283 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073b4cbd-e998-4196-a338-8fb1b3ff8207 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:49.056116 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:49.071532 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bd2305-8922-4524-bb90-7fdd990ee18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:49.072165 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd2305-8922-4524-bb90-7fdd990ee18b None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:49.072451 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd2305-8922-4524-bb90-7fdd990ee18b None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:49.072757 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd2305-8922-4524-bb90-7fdd990ee18b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:49.078486 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9568ef1-691e-4eed-9ca7-a9577818f414 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:49.085247 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bd2305-8922-4524-bb90-7fdd990ee18b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:49.086717 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bd2305-8922-4524-bb90-7fdd990ee18b None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:49.104367 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-e0bd2305-8922-4524-bb90-7fdd990ee18b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:49.104367 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:49.104367 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:49.104367 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:49.104367 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:49.104367 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:49.104367 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:49.104367 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:49.104367 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:49.104367 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:49.104367 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:49.104367 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:49.104367 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:49.104367 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:49.104367 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:49.104367 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:49.105345 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:49.105345 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:49.105345 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:49.105345 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:49.105345 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:49.105345 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:49.105345 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:49.105345 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:49.105345 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:49.105345 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:49.105345 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:49.105345 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:49.105345 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:49.105345 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:49.105345 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:49.105345 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:49.105345 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:49.106212 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:49.106212 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:49.106212 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:49.106212 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 376/740] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:49.107002 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79322b80-5d9a-49a5-b8cb-a0b4f5052a77 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:49.107292 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79322b80-5d9a-49a5-b8cb-a0b4f5052a77 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:49.107574 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79322b80-5d9a-49a5-b8cb-a0b4f5052a77 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:49.109162 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 377/741] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:49.109847 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7fbd66-3e51-479d-91fd-bd489d401afa None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:49.110123 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7fbd66-3e51-479d-91fd-bd489d401afa None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:49.110422 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7fbd66-3e51-479d-91fd-bd489d401afa None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:49.113109 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 378/742] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:49.113477 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7fbd66-3e51-479d-91fd-bd489d401afa None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:49.132194 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4705dec8-f0bb-403d-9972-b375dcca59a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:49.132956 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705dec8-f0bb-403d-9972-b375dcca59a2 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:49.133306 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705dec8-f0bb-403d-9972-b375dcca59a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:49.133650 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705dec8-f0bb-403d-9972-b375dcca59a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:49.166084 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:49.210926 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4705dec8-f0bb-403d-9972-b375dcca59a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:49.212850 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4705dec8-f0bb-403d-9972-b375dcca59a2 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:49.238078 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7fbd66-3e51-479d-91fd-bd489d401afa None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:49.274990 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8898612-b0cc-44b8-8610-1ef1fb8c522c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:49.275903 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8898612-b0cc-44b8-8610-1ef1fb8c522c None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:49.276375 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8898612-b0cc-44b8-8610-1ef1fb8c522c None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:49.276681 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8898612-b0cc-44b8-8610-1ef1fb8c522c None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:49.277377 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8898612-b0cc-44b8-8610-1ef1fb8c522c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:49.277685 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8898612-b0cc-44b8-8610-1ef1fb8c522c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:49.279083 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8898612-b0cc-44b8-8610-1ef1fb8c522c None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:49.291936 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 379/743] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:49.293199 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd7975-0965-47a3-98e8-8d1a0772a401 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:49.293485 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd7975-0965-47a3-98e8-8d1a0772a401 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:49.293761 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd7975-0965-47a3-98e8-8d1a0772a401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:49.312922 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-67bd7975-0965-47a3-98e8-8d1a0772a401 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:49.314575 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-67bd7975-0965-47a3-98e8-8d1a0772a401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:49.326845 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 365/744] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:48:48 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 04 00:48:49.328161 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506c63d-38c1-41e8-abb1-a551788e174b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:49.328449 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506c63d-38c1-41e8-abb1-a551788e174b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:49.328767 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506c63d-38c1-41e8-abb1-a551788e174b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:49.347586 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-4506c63d-38c1-41e8-abb1-a551788e174b None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:49.348553 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:49.348799 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4506c63d-38c1-41e8-abb1-a551788e174b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:49.387562 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 380/745] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:49 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:49.390320 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd7975-0965-47a3-98e8-8d1a0772a401 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:49.409044 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:49.422991 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 366/746] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:49 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 04 00:48:49.422991 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506c63d-38c1-41e8-abb1-a551788e174b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:49.568198 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd7975-0965-47a3-98e8-8d1a0772a401 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:49.575275 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506c63d-38c1-41e8-abb1-a551788e174b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:49.616261 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115d221-5997-449d-852a-c73170434f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:49.616920 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115d221-5997-449d-852a-c73170434f0c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:49.617196 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115d221-5997-449d-852a-c73170434f0c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:49.617463 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115d221-5997-449d-852a-c73170434f0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:49.622733 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248170f-aa75-4dbd-87a7-18ee5fde5a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:49.623600 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248170f-aa75-4dbd-87a7-18ee5fde5a12 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:49.624089 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248170f-aa75-4dbd-87a7-18ee5fde5a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:49.624697 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248170f-aa75-4dbd-87a7-18ee5fde5a12 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:49.628872 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115d221-5997-449d-852a-c73170434f0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:49.630304 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115d221-5997-449d-852a-c73170434f0c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:49.637748 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7248170f-aa75-4dbd-87a7-18ee5fde5a12 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:49.640778 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7248170f-aa75-4dbd-87a7-18ee5fde5a12 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:49.645939 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-9115d221-5997-449d-852a-c73170434f0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:49.645939 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:49.645939 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:49.645939 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:49.645939 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:49.645939 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:49.645939 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:49.645939 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:49.645939 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:49.645939 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:49.645939 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:49.645939 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:49.645939 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:49.645939 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:49.645939 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:49.645939 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:49.646965 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:49.646965 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:49.646965 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:49.646965 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:49.646965 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:49.646965 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:49.646965 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:49.646965 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:49.646965 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:49.646965 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:49.646965 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:49.646965 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:49.646965 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:49.646965 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:49.646965 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:49.646965 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:49.646965 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:49.647847 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:49.647847 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:49.647847 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:49.652186 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 367/747] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:49.653493 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115d221-5997-449d-852a-c73170434f0c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:49.658264 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-7248170f-aa75-4dbd-87a7-18ee5fde5a12 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:49.658264 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:49.658264 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:49.658264 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:49.658264 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:49.658264 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:49.658264 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:49.658264 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:49.658264 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:49.658264 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:49.658264 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:49.658264 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:49.658264 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:48:49.658264 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:48:49.658264 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:48:49.659148 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:48:49.659148 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:49.659148 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:49.659148 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:49.659148 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:49.659148 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:49.659148 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:49.659148 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:49.659148 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:49.659148 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:49.659148 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:49.659148 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:49.659148 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:49.659148 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:49.659148 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:49.659148 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:49.659148 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:49.660026 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:48:49.660026 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:48:49.660026 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:48:49.660026 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:48:49.660026 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:48:49.660026 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:48:49.660026 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:49.660026 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:49.662387 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 381/748] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:48:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:49.679219 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248170f-aa75-4dbd-87a7-18ee5fde5a12 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:49.713183 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:49.717695 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:49.761105 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115d221-5997-449d-852a-c73170434f0c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:49.798165 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbc4888-f4b2-4648-ac95-3144babe5aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:49.799121 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc4888-f4b2-4648-ac95-3144babe5aa9 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:49.799121 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc4888-f4b2-4648-ac95-3144babe5aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:49.799504 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc4888-f4b2-4648-ac95-3144babe5aa9 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:49.800153 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbc4888-f4b2-4648-ac95-3144babe5aa9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:49.800477 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbc4888-f4b2-4648-ac95-3144babe5aa9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:49.801845 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbc4888-f4b2-4648-ac95-3144babe5aa9 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:49.802160 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248170f-aa75-4dbd-87a7-18ee5fde5a12 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:49.816491 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 368/749] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:49.816491 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbc4888-f4b2-4648-ac95-3144babe5aa9 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:49.843419 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c4cf9-cf85-4c3b-b083-f5622a1cbfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:49.844020 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c4cf9-cf85-4c3b-b083-f5622a1cbfac None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:49.844347 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c4cf9-cf85-4c3b-b083-f5622a1cbfac None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:49.844651 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c4cf9-cf85-4c3b-b083-f5622a1cbfac None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:49.845512 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5c4cf9-cf85-4c3b-b083-f5622a1cbfac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:49.845817 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5c4cf9-cf85-4c3b-b083-f5622a1cbfac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:49.847245 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5c4cf9-cf85-4c3b-b083-f5622a1cbfac None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:49.861126 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 382/750] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:48:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:49.861527 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c4cf9-cf85-4c3b-b083-f5622a1cbfac None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:49.930372 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbc4888-f4b2-4648-ac95-3144babe5aa9 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:49.933729 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:49.957307 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:49.971179 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce099ae4-6f9d-471a-9dff-aa842bdc9468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:49.971811 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce099ae4-6f9d-471a-9dff-aa842bdc9468 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:49.972091 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce099ae4-6f9d-471a-9dff-aa842bdc9468 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:49.972388 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce099ae4-6f9d-471a-9dff-aa842bdc9468 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:49.973491 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c4cf9-cf85-4c3b-b083-f5622a1cbfac None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:49.982795 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce099ae4-6f9d-471a-9dff-aa842bdc9468 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:49.988083 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-ce099ae4-6f9d-471a-9dff-aa842bdc9468 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:48:49.988083 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:49.988083 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:49.988083 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:49.988083 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:49.988083 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:49.988083 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:49.988083 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:49.988083 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:49.988083 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:49.988083 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:49.988083 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:49.988083 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:49.988083 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:49.988083 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:49.988083 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:49.989050 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:49.989050 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:49.989050 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:49.989050 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:49.989050 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:49.989050 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:49.989050 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:49.989050 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:49.989050 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:49.989050 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:49.989050 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:49.989050 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:49.989050 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:49.989050 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:49.989050 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:49.989050 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:49.989871 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:49.989871 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:49.989871 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:49.989871 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:49.989871 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:49.989871 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:49.989871 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:49.989871 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:49.989871 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:49.989871 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:49.989871 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:48:49.989871 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:49.989871 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 369/751] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:48:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:49.991140 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce099ae4-6f9d-471a-9dff-aa842bdc9468 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:50.014403 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59869be-02d0-4457-a7fc-3dfd50d54fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:50.015017 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59869be-02d0-4457-a7fc-3dfd50d54fd5 None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:50.015298 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59869be-02d0-4457-a7fc-3dfd50d54fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:50.015570 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59869be-02d0-4457-a7fc-3dfd50d54fd5 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:50.016220 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59869be-02d0-4457-a7fc-3dfd50d54fd5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:50.017674 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59869be-02d0-4457-a7fc-3dfd50d54fd5 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:50.091090 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 383/752] 10.222.0.27 () {60 vars in 1141 bytes} [Sun Sep 4 00:48:49 2022] POST /identity/v3/services => generated 251 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:48:50.092979 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59869be-02d0-4457-a7fc-3dfd50d54fd5 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:50.107708 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce099ae4-6f9d-471a-9dff-aa842bdc9468 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:50.157168 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9515d97-01df-4559-8c3e-a849796a1457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:50.158812 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9515d97-01df-4559-8c3e-a849796a1457 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:50.159097 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9515d97-01df-4559-8c3e-a849796a1457 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:50.159371 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9515d97-01df-4559-8c3e-a849796a1457 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:50.167841 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9515d97-01df-4559-8c3e-a849796a1457 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:50.169420 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9515d97-01df-4559-8c3e-a849796a1457 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:50.183613 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-c9515d97-01df-4559-8c3e-a849796a1457 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:50.183613 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:50.183613 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:50.183613 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:50.183613 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:50.183613 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:50.183613 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:50.183613 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:50.183613 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:50.183613 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:50.183613 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:50.183613 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:50.183613 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:48:50.183613 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:48:50.183613 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:48:50.184556 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:48:50.184556 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:50.184556 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:50.184556 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:50.184556 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:50.184556 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:50.184556 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:50.184556 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:50.184556 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:50.184556 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:50.184556 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:50.184556 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:50.184556 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:50.184556 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:50.184556 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:50.184556 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:50.184556 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:50.185430 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:48:50.185430 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:48:50.185430 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:48:50.185430 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:48:50.185430 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:48:50.185430 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:48:50.185430 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:50.185430 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:50.190130 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 370/753] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:48:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:50.191107 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9515d97-01df-4559-8c3e-a849796a1457 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:50.215290 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:50.267010 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59869be-02d0-4457-a7fc-3dfd50d54fd5 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:50.315018 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10b87f7-0b29-4f0e-a4f3-1c1132fee5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:50.315848 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b87f7-0b29-4f0e-a4f3-1c1132fee5e2 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:50.316135 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b87f7-0b29-4f0e-a4f3-1c1132fee5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:50.316411 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b87f7-0b29-4f0e-a4f3-1c1132fee5e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:50.317284 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10b87f7-0b29-4f0e-a4f3-1c1132fee5e2 None admin] RBAC: Adding query filter params (role.id=6e3e398ba29e436e8532a45d2fdf1321, scope.project.id=5100a8d316d243fb8f82e9044e1b3b06, user.id=d6d01ca673a1404c9b4cea51036c595c) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:50.317708 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10b87f7-0b29-4f0e-a4f3-1c1132fee5e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:50.319142 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10b87f7-0b29-4f0e-a4f3-1c1132fee5e2 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:50.320668 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9515d97-01df-4559-8c3e-a849796a1457 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:50.325397 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 384/754] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:48:50 2022] GET /identity/v3/role_assignments?user.id=d6d01ca673a1404c9b4cea51036c595c&scope.project.id=5100a8d316d243fb8f82e9044e1b3b06&role.id=6e3e398ba29e436e8532a45d2fdf1321 => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:50.326368 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10b87f7-0b29-4f0e-a4f3-1c1132fee5e2 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:50.340011 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:50.370077 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d98ae-d33f-4497-bc1c-1c23779e4208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:50.371100 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d98ae-d33f-4497-bc1c-1c23779e4208 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:50.371570 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d98ae-d33f-4497-bc1c-1c23779e4208 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:50.372018 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d98ae-d33f-4497-bc1c-1c23779e4208 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:50.372899 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4d98ae-d33f-4497-bc1c-1c23779e4208 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:50.373402 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4d98ae-d33f-4497-bc1c-1c23779e4208 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:50.375571 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4d98ae-d33f-4497-bc1c-1c23779e4208 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:50.394233 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 371/755] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:48:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:50.401426 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d98ae-d33f-4497-bc1c-1c23779e4208 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:50.533431 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:50.534720 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10b87f7-0b29-4f0e-a4f3-1c1132fee5e2 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:50.538941 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d98ae-d33f-4497-bc1c-1c23779e4208 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:50.602799 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf6cc5-a450-48f8-b819-a4e6f1336fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:50.603415 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf6cc5-a450-48f8-b819-a4e6f1336fda None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:50.603693 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf6cc5-a450-48f8-b819-a4e6f1336fda None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:50.603965 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf6cc5-a450-48f8-b819-a4e6f1336fda None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:50.604689 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cf6cc5-a450-48f8-b819-a4e6f1336fda None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:50.604990 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cf6cc5-a450-48f8-b819-a4e6f1336fda None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:50.606423 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cf6cc5-a450-48f8-b819-a4e6f1336fda None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:50.611115 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e05ebb-9e0f-43b6-bc2a-f2bfcdd19d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:50.611753 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e05ebb-9e0f-43b6-bc2a-f2bfcdd19d18 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:50.612030 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e05ebb-9e0f-43b6-bc2a-f2bfcdd19d18 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:50.612301 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e05ebb-9e0f-43b6-bc2a-f2bfcdd19d18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:50.629121 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e05ebb-9e0f-43b6-bc2a-f2bfcdd19d18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:50.630930 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e05ebb-9e0f-43b6-bc2a-f2bfcdd19d18 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:50.631040 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 385/756] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:48:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:50.641482 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf6cc5-a450-48f8-b819-a4e6f1336fda None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:50.649399 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-09e05ebb-9e0f-43b6-bc2a-f2bfcdd19d18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:50.649399 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:50.649399 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:50.649399 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:50.649399 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:50.649399 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:50.649399 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:50.649399 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:50.649399 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:50.649399 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:50.649399 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:50.649399 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:50.649399 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:50.649399 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:50.649399 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:50.649399 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:50.650329 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:50.650329 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:50.650329 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:50.650329 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:50.650329 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:50.650329 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:50.650329 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:50.650329 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:50.650329 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:50.650329 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:50.650329 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:50.650329 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:50.650329 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:50.650329 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:50.650329 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:50.650329 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:50.650329 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:50.651206 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:50.651206 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:50.651206 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:50.653308 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 372/757] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:50.669350 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e05ebb-9e0f-43b6-bc2a-f2bfcdd19d18 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:50.723221 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:50.758524 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf6cc5-a450-48f8-b819-a4e6f1336fda None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:50.805169 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042cabe-8461-49e4-8b70-7342cf7ca174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:50.806160 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042cabe-8461-49e4-8b70-7342cf7ca174 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:50.806654 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042cabe-8461-49e4-8b70-7342cf7ca174 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:50.808348 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042cabe-8461-49e4-8b70-7342cf7ca174 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:50.809302 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042cabe-8461-49e4-8b70-7342cf7ca174 None admin] RBAC: Adding query filter params (role.id=c8a59d34ee2f4cefa6940cac778260c7, scope.project.id=5100a8d316d243fb8f82e9044e1b3b06, user.id=2297d8b3fe8e47eb9fd39a6e3eea8f54) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:50.811420 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042cabe-8461-49e4-8b70-7342cf7ca174 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:50.813089 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042cabe-8461-49e4-8b70-7342cf7ca174 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:50.824299 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 386/758] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:48:50 2022] GET /identity/v3/role_assignments?user.id=2297d8b3fe8e47eb9fd39a6e3eea8f54&scope.project.id=5100a8d316d243fb8f82e9044e1b3b06&role.id=c8a59d34ee2f4cefa6940cac778260c7 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:50.874510 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e05ebb-9e0f-43b6-bc2a-f2bfcdd19d18 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:50.918556 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:50.930908 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc37187-f352-4135-a984-c9ffadc5836e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:50.931531 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc37187-f352-4135-a984-c9ffadc5836e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:50.931812 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc37187-f352-4135-a984-c9ffadc5836e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:50.932079 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc37187-f352-4135-a984-c9ffadc5836e None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:50.932749 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc37187-f352-4135-a984-c9ffadc5836e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:50.933049 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc37187-f352-4135-a984-c9ffadc5836e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:50.934451 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc37187-f352-4135-a984-c9ffadc5836e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:50.954789 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 373/759] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:48:50.960918 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042cabe-8461-49e4-8b70-7342cf7ca174 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:51.125012 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042cabe-8461-49e4-8b70-7342cf7ca174 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:51.174187 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:51.179791 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57d718-c88c-40ad-a569-9354ecb7e881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:51.180427 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57d718-c88c-40ad-a569-9354ecb7e881 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:51.180735 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57d718-c88c-40ad-a569-9354ecb7e881 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:51.181011 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57d718-c88c-40ad-a569-9354ecb7e881 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:51.186201 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ca617-7f33-474d-a51c-c69fe494e7a3 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:51.186486 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ca617-7f33-474d-a51c-c69fe494e7a3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:51.189950 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ca617-7f33-474d-a51c-c69fe494e7a3 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:51.189950 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 374/760] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:51.195089 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5002098-db72-4bf1-9824-33ae0c6bf1e0 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:51.195373 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5002098-db72-4bf1-9824-33ae0c6bf1e0 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:51.195648 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5002098-db72-4bf1-9824-33ae0c6bf1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:51.198318 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d57d718-c88c-40ad-a569-9354ecb7e881 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:51.199776 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d57d718-c88c-40ad-a569-9354ecb7e881 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:51.216744 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f5002098-db72-4bf1-9824-33ae0c6bf1e0 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:51.218046 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f5002098-db72-4bf1-9824-33ae0c6bf1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:51.223976 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-0d57d718-c88c-40ad-a569-9354ecb7e881 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 04 00:48:51.223976 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:51.223976 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:51.223976 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:51.223976 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:51.223976 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:51.223976 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:51.223976 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:51.223976 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:51.223976 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:51.223976 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:51.223976 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:51.223976 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:48:51.223976 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:48:51.223976 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:48:51.223976 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:48:51.224965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:51.224965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:51.224965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:48:51.224965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:51.224965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:48:51.224965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:51.224965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:51.224965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:51.224965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:51.224965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:51.224965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:51.224965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:51.224965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:51.224965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:51.224965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:51.224965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:51.224965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:51.224965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:51.225906 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:51.225906 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:51.225906 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:48:51.225906 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:48:51.225906 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:48:51.225906 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:48:51.225906 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:48:51.225906 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:48:51.225906 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 04 00:48:51.225906 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:51.240492 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 387/761] 10.222.0.27 () {56 vars in 1093 bytes} [Sun Sep 4 00:48:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:51.244301 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57d718-c88c-40ad-a569-9354ecb7e881 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:51.351517 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 375/762] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 00:48:51.397014 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57d718-c88c-40ad-a569-9354ecb7e881 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:51.432355 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de48c82-e758-458d-916b-97b414d0e430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:51.432990 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de48c82-e758-458d-916b-97b414d0e430 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:51.433269 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de48c82-e758-458d-916b-97b414d0e430 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:51.433540 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de48c82-e758-458d-916b-97b414d0e430 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:51.434264 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de48c82-e758-458d-916b-97b414d0e430 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:51.434565 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de48c82-e758-458d-916b-97b414d0e430 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:51.436003 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de48c82-e758-458d-916b-97b414d0e430 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:51.455503 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 388/763] 10.222.0.27 () {56 vars in 1133 bytes} [Sun Sep 4 00:48:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:48:51.461087 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5002098-db72-4bf1-9824-33ae0c6bf1e0 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:51.591992 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5002098-db72-4bf1-9824-33ae0c6bf1e0 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:51.627583 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:51.638841 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1b966-7730-4e61-9909-d5bf12c0c723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:51.643434 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1b966-7730-4e61-9909-d5bf12c0c723 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:51.643434 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1b966-7730-4e61-9909-d5bf12c0c723 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:51.643434 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1b966-7730-4e61-9909-d5bf12c0c723 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:51.653022 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b1b966-7730-4e61-9909-d5bf12c0c723 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:51.654392 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b1b966-7730-4e61-9909-d5bf12c0c723 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:51.679775 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-05b1b966-7730-4e61-9909-d5bf12c0c723 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:51.679775 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:51.679775 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:51.679775 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:51.679775 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:51.679775 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:51.679775 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:51.679775 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:51.679775 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:51.679775 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:51.679775 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:51.679775 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:51.679775 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:51.679775 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:51.679775 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:51.679775 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:51.680781 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:51.680781 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:51.680781 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:51.680781 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:51.680781 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:51.680781 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:51.680781 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:51.680781 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:51.680781 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:51.680781 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:51.680781 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:51.680781 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:51.680781 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:51.680781 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:51.680781 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:51.680781 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:51.680781 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:51.681656 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:51.681656 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:51.681656 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:51.681656 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 376/764] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:51.687916 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de48c82-e758-458d-916b-97b414d0e430 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:51.699287 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1b966-7730-4e61-9909-d5bf12c0c723 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:51.781667 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:51.867002 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de48c82-e758-458d-916b-97b414d0e430 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:51.884420 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1b966-7730-4e61-9909-d5bf12c0c723 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:51.930906 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9153935-06bd-4c8f-974a-b1a0f883f76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:51.931442 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9153935-06bd-4c8f-974a-b1a0f883f76b None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:51.931724 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9153935-06bd-4c8f-974a-b1a0f883f76b None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:51.932002 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9153935-06bd-4c8f-974a-b1a0f883f76b None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:51.932677 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9153935-06bd-4c8f-974a-b1a0f883f76b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:51.932984 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9153935-06bd-4c8f-974a-b1a0f883f76b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:51.934373 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9153935-06bd-4c8f-974a-b1a0f883f76b None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:51.949284 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39971fd9-ebbc-4bc2-b6dd-3ddd905129fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:51.949915 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39971fd9-ebbc-4bc2-b6dd-3ddd905129fa None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:51.950196 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39971fd9-ebbc-4bc2-b6dd-3ddd905129fa None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:51.950467 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39971fd9-ebbc-4bc2-b6dd-3ddd905129fa None admin] PATH_INFO: `/v3/services/network` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:51.951054 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39971fd9-ebbc-4bc2-b6dd-3ddd905129fa None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:51.952406 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39971fd9-ebbc-4bc2-b6dd-3ddd905129fa None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:51.956892 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 389/765] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:51.962567 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9153935-06bd-4c8f-974a-b1a0f883f76b None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:51.968807 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-39971fd9-ebbc-4bc2-b6dd-3ddd905129fa None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 04 00:48:51.968807 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:51.968807 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 00:48:51.968807 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 00:48:51.968807 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 00:48:51.968807 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 00:48:51.968807 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 00:48:51.968807 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:48:51.968807 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 04 00:48:51.968807 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:51.968807 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 00:48:51.968807 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:51.968807 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:51.968807 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:51.968807 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:51.968807 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:51.968807 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:51.969847 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:51.969847 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:51.969847 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:51.969847 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:51.969847 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:51.969847 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:51.969847 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 00:48:51.969847 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 00:48:51.969847 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 00:48:51.969847 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 00:48:51.969847 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:51.969847 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:51.969847 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:51.969847 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:51.969847 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:51.969847 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:51.970878 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:51.970878 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:51.970878 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:51.970878 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:51.970878 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:51.970878 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:51.970878 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:51.970878 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:51.970878 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:51.970878 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:51.970878 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 00:48:51.970878 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:48:51.970878 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 04 00:48:51.970878 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:51.973390 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 377/766] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:48:51 2022] GET /identity/v3/services/network => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:51.979816 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39971fd9-ebbc-4bc2-b6dd-3ddd905129fa None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:52.027225 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:52.117759 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39971fd9-ebbc-4bc2-b6dd-3ddd905129fa None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:52.160971 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9153935-06bd-4c8f-974a-b1a0f883f76b None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:52.171909 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:52.197722 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec00a5-b0fe-44ef-8598-b6cd75f11e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:52.198055 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1fb274-7217-44b7-9610-7353f58dcce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:52.198341 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec00a5-b0fe-44ef-8598-b6cd75f11e9e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:52.198602 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec00a5-b0fe-44ef-8598-b6cd75f11e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:52.198704 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fb274-7217-44b7-9610-7353f58dcce1 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:52.198932 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec00a5-b0fe-44ef-8598-b6cd75f11e9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:52.199046 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fb274-7217-44b7-9610-7353f58dcce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:52.199296 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fb274-7217-44b7-9610-7353f58dcce1 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:52.199936 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1fb274-7217-44b7-9610-7353f58dcce1 None admin] RBAC: Adding query filter params (name=network) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:52.200239 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1fb274-7217-44b7-9610-7353f58dcce1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:52.201634 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1fb274-7217-44b7-9610-7353f58dcce1 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:52.206296 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ec00a5-b0fe-44ef-8598-b6cd75f11e9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:52.207738 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ec00a5-b0fe-44ef-8598-b6cd75f11e9e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:52.218801 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 378/767] 10.222.0.27 () {56 vars in 1112 bytes} [Sun Sep 4 00:48:51 2022] GET /identity/v3/services?name=network => generated 124 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:52.222482 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-b7ec00a5-b0fe-44ef-8598-b6cd75f11e9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:52.222482 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:52.222482 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:52.222482 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:52.222482 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:52.222482 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:52.222482 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:52.222482 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:52.222482 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:52.222482 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:52.222482 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:52.222482 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:52.222482 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:48:52.222482 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:48:52.222482 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:48:52.223362 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:48:52.223362 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:52.223362 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:52.223362 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:52.223362 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:52.223362 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:52.223362 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:52.223362 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:52.223362 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:52.223362 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:52.223362 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:52.223362 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:52.223362 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:52.223362 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:52.223362 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:52.223362 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:52.223362 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:52.224262 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:48:52.224262 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:48:52.224262 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:48:52.224262 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:48:52.224262 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:48:52.224262 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:48:52.224262 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:52.224262 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:52.225400 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 390/768] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:48:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:48:52.230602 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1fb274-7217-44b7-9610-7353f58dcce1 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:52.232058 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec00a5-b0fe-44ef-8598-b6cd75f11e9e None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:52.428791 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec00a5-b0fe-44ef-8598-b6cd75f11e9e None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:52.433744 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1fb274-7217-44b7-9610-7353f58dcce1 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:52.435441 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:52.466307 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4bbdd5-b62e-491f-b95b-74b7474c4e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:52.466838 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4bbdd5-b62e-491f-b95b-74b7474c4e50 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:52.467119 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4bbdd5-b62e-491f-b95b-74b7474c4e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:52.467395 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4bbdd5-b62e-491f-b95b-74b7474c4e50 None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:52.468086 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4bbdd5-b62e-491f-b95b-74b7474c4e50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:52.468390 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4bbdd5-b62e-491f-b95b-74b7474c4e50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:52.470569 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4bbdd5-b62e-491f-b95b-74b7474c4e50 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:52.480451 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b36890-bf95-44e9-a257-a6068b10ced3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:52.481090 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b36890-bf95-44e9-a257-a6068b10ced3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:52.481371 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b36890-bf95-44e9-a257-a6068b10ced3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:52.481637 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b36890-bf95-44e9-a257-a6068b10ced3 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:52.482259 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b36890-bf95-44e9-a257-a6068b10ced3 None admin] RBAC: Adding query filter params (type=network) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:52.482560 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b36890-bf95-44e9-a257-a6068b10ced3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:52.483234 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 391/769] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:48:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:52.488495 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b36890-bf95-44e9-a257-a6068b10ced3 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:52.488624 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff2bf6-c3dc-442a-8e7a-752f308b6d10 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:52.488624 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff2bf6-c3dc-442a-8e7a-752f308b6d10 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:52.488624 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff2bf6-c3dc-442a-8e7a-752f308b6d10 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:52.488624 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 392/770] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:52.488624 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca85fc2-1e61-4535-9871-7dcb0b44ca14 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:52.488624 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca85fc2-1e61-4535-9871-7dcb0b44ca14 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:52.489023 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca85fc2-1e61-4535-9871-7dcb0b44ca14 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:52.489956 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 393/771] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:52.490692 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca85fc2-1e61-4535-9871-7dcb0b44ca14 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:52.494778 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 379/772] 10.222.0.27 () {56 vars in 1112 bytes} [Sun Sep 4 00:48:52 2022] GET /identity/v3/services?type=network => generated 350 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:52.495709 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebb398-08ad-4248-b65b-0a7885262ed6 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:52.496014 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebb398-08ad-4248-b65b-0a7885262ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:52.496288 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebb398-08ad-4248-b65b-0a7885262ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:52.514591 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-e0ebb398-08ad-4248-b65b-0a7885262ed6 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:52.515584 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e0ebb398-08ad-4248-b65b-0a7885262ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:52.526859 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:52.589222 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 380/773] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:48:52.590539 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166cc25-1980-4a54-8f2b-55a7ca2ad185 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:52.590828 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166cc25-1980-4a54-8f2b-55a7ca2ad185 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:52.591107 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166cc25-1980-4a54-8f2b-55a7ca2ad185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:52.608907 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca85fc2-1e61-4535-9871-7dcb0b44ca14 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:52.612559 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-d166cc25-1980-4a54-8f2b-55a7ca2ad185 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:52.613897 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d166cc25-1980-4a54-8f2b-55a7ca2ad185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:52.649426 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ecc93-1059-4ee8-826f-fc843785f6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:52.650399 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ecc93-1059-4ee8-826f-fc843785f6af None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:52.650881 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ecc93-1059-4ee8-826f-fc843785f6af None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:52.651432 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ecc93-1059-4ee8-826f-fc843785f6af None admin] PATH_INFO: `/v3/projects/5100a8d316d243fb8f82e9044e1b3b06/users/68bc00bda5194cd8a7928f8c4df4ccd1/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:52.653676 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154ecc93-1059-4ee8-826f-fc843785f6af None admin] RBAC: Authorizing `identity:create_grant(project_id=5100a8d316d243fb8f82e9044e1b3b06, user_id=68bc00bda5194cd8a7928f8c4df4ccd1, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:52.655411 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154ecc93-1059-4ee8-826f-fc843785f6af None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:52.693692 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 381/774] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:48:52.694914 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d166cc25-1980-4a54-8f2b-55a7ca2ad185 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:52.730320 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 394/775] 10.222.0.27 () {58 vars in 1439 bytes} [Sun Sep 4 00:48:52 2022] PUT /identity/v3/projects/5100a8d316d243fb8f82e9044e1b3b06/users/68bc00bda5194cd8a7928f8c4df4ccd1/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 00:48:52.731005 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:52.817376 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d166cc25-1980-4a54-8f2b-55a7ca2ad185 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:52.866450 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ceb6e8-168e-4def-bb4d-2f77aab9aaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:52.867396 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ceb6e8-168e-4def-bb4d-2f77aab9aaae None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:52.867857 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ceb6e8-168e-4def-bb4d-2f77aab9aaae None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:52.868335 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ceb6e8-168e-4def-bb4d-2f77aab9aaae None admin] PATH_INFO: `/v3/endpoints` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:52.869262 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ceb6e8-168e-4def-bb4d-2f77aab9aaae None admin] RBAC: Adding query filter params (interface=public, service_id=a0c22a9a85f74e53be111e588fa3b5e1, region_id=RegionOne) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:52.869752 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ceb6e8-168e-4def-bb4d-2f77aab9aaae None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:52.871284 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ceb6e8-168e-4def-bb4d-2f77aab9aaae None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:52.880160 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 382/776] 10.222.0.27 () {56 vars in 1251 bytes} [Sun Sep 4 00:48:52 2022] GET /identity/v3/endpoints?service_id=a0c22a9a85f74e53be111e588fa3b5e1&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:52.885583 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ecc93-1059-4ee8-826f-fc843785f6af None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:52.887077 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ceb6e8-168e-4def-bb4d-2f77aab9aaae None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:53.008868 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:53.012529 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ecc93-1059-4ee8-826f-fc843785f6af None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:53.058346 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:53.060356 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a8f10c-f218-4b14-ba55-cd588eacda3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:53.061072 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8f10c-f218-4b14-ba55-cd588eacda3a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:53.061355 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8f10c-f218-4b14-ba55-cd588eacda3a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:53.061627 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8f10c-f218-4b14-ba55-cd588eacda3a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:53.062355 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a8f10c-f218-4b14-ba55-cd588eacda3a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:53.064046 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a8f10c-f218-4b14-ba55-cd588eacda3a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:53.070504 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ceb6e8-168e-4def-bb4d-2f77aab9aaae None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:53.074143 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-c9a8f10c-f218-4b14-ba55-cd588eacda3a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 04 00:48:53.074143 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:53.074143 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 00:48:53.074143 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 00:48:53.074143 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 00:48:53.074143 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 00:48:53.074143 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 00:48:53.074143 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:48:53.074143 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 04 00:48:53.074143 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:53.074143 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 00:48:53.074143 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:53.074143 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:53.074143 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:53.074143 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:53.074143 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:53.074143 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:53.075113 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:53.075113 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:53.075113 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:53.075113 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:53.075113 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:53.075113 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:53.075113 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 00:48:53.075113 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 00:48:53.075113 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 00:48:53.075113 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 00:48:53.075113 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:53.075113 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:53.075113 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:53.075113 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:53.075113 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:53.075113 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:53.075936 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:53.075936 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:53.075936 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:53.075936 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:53.075936 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:53.075936 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:53.075936 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:53.075936 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:53.075936 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:53.075936 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:53.075936 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 00:48:53.075936 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:48:53.075936 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 04 00:48:53.075936 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:53.075936 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 395/777] 10.222.0.27 () {56 vars in 1129 bytes} [Sun Sep 4 00:48:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:53.076761 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a8f10c-f218-4b14-ba55-cd588eacda3a None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:53.117730 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1378de-1f97-44a3-8436-7e7d13108148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:53.118867 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1378de-1f97-44a3-8436-7e7d13108148 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:53.120143 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1378de-1f97-44a3-8436-7e7d13108148 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:53.120688 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1378de-1f97-44a3-8436-7e7d13108148 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:53.121535 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1378de-1f97-44a3-8436-7e7d13108148 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:53.123130 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1378de-1f97-44a3-8436-7e7d13108148 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:53.130884 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 383/778] 10.222.0.27 () {56 vars in 1087 bytes} [Sun Sep 4 00:48:52 2022] GET /identity/v3/services => generated 772 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:53.137005 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1378de-1f97-44a3-8436-7e7d13108148 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:53.252689 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a8f10c-f218-4b14-ba55-cd588eacda3a None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:53.294644 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4bd654-20f3-4bbd-bfea-3b68e9bdb2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:53.295531 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4bd654-20f3-4bbd-bfea-3b68e9bdb2c8 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:53.295959 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4bd654-20f3-4bbd-bfea-3b68e9bdb2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:53.296491 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4bd654-20f3-4bbd-bfea-3b68e9bdb2c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:53.313527 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4bd654-20f3-4bbd-bfea-3b68e9bdb2c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:53.313802 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1378de-1f97-44a3-8436-7e7d13108148 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:53.318871 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-8a4bd654-20f3-4bbd-bfea-3b68e9bdb2c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:53.318871 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:53.318871 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:53.318871 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:53.318871 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:53.318871 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:53.318871 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:53.318871 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:53.318871 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:53.318871 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:53.318871 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:53.318871 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:53.318871 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:53.318871 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:53.318871 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:53.318871 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:53.319827 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:53.319827 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:53.319827 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:53.319827 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:53.319827 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:53.319827 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:53.319827 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:53.319827 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:53.319827 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:53.319827 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:53.319827 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:53.319827 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:53.319827 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:53.319827 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:53.319827 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:53.319827 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:53.320691 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:53.320691 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:53.320691 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:53.320691 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:53.320691 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:53.320691 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:53.320691 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:53.320691 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:53.320691 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:53.320691 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:53.320691 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:53.320691 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:53.320691 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 396/779] 10.222.0.27 () {56 vars in 1102 bytes} [Sun Sep 4 00:48:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:53.326882 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67fc63-d748-4975-8cc8-7febe40f724f None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:53.326882 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67fc63-d748-4975-8cc8-7febe40f724f None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:53.326882 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67fc63-d748-4975-8cc8-7febe40f724f None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:53.329540 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 397/780] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:53.330750 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e67fc63-d748-4975-8cc8-7febe40f724f None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:53.353322 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:53.361396 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa7222-6e90-4198-bbd2-fbdd44ac8104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:53.362084 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa7222-6e90-4198-bbd2-fbdd44ac8104 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:53.362458 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa7222-6e90-4198-bbd2-fbdd44ac8104 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:53.362792 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa7222-6e90-4198-bbd2-fbdd44ac8104 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:53.363486 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa7222-6e90-4198-bbd2-fbdd44ac8104 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:53.363854 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa7222-6e90-4198-bbd2-fbdd44ac8104 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:53.365392 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa7222-6e90-4198-bbd2-fbdd44ac8104 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:53.370617 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 384/781] 10.222.0.27 () {56 vars in 1124 bytes} [Sun Sep 4 00:48:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:53.371964 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43adc2-6685-4e6c-a426-5cbd036eaeba None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:53.374703 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43adc2-6685-4e6c-a426-5cbd036eaeba None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:53.374703 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43adc2-6685-4e6c-a426-5cbd036eaeba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:53.392769 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-6d43adc2-6685-4e6c-a426-5cbd036eaeba None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:53.394015 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6d43adc2-6685-4e6c-a426-5cbd036eaeba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:53.408946 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:53.445898 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e67fc63-d748-4975-8cc8-7febe40f724f None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:53.481161 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 385/782] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:48:53.482036 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d43adc2-6685-4e6c-a426-5cbd036eaeba None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:53.488273 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ece07b-a72c-4a3e-8539-0f44c6fb8643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:53.488942 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ece07b-a72c-4a3e-8539-0f44c6fb8643 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:53.489223 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ece07b-a72c-4a3e-8539-0f44c6fb8643 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:53.489499 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ece07b-a72c-4a3e-8539-0f44c6fb8643 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:53.490230 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ece07b-a72c-4a3e-8539-0f44c6fb8643 None admin] RBAC: Adding query filter params (name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:53.490538 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ece07b-a72c-4a3e-8539-0f44c6fb8643 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:53.492026 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ece07b-a72c-4a3e-8539-0f44c6fb8643 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:53.519600 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 398/783] 10.222.0.27 () {56 vars in 1103 bytes} [Sun Sep 4 00:48:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:53.525030 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ece07b-a72c-4a3e-8539-0f44c6fb8643 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:53.608373 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d43adc2-6685-4e6c-a426-5cbd036eaeba None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:53.647922 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4ace8-aa6a-4449-8a65-6837b292cd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:53.648585 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4ace8-aa6a-4449-8a65-6837b292cd9b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:53.648870 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4ace8-aa6a-4449-8a65-6837b292cd9b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:53.649145 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4ace8-aa6a-4449-8a65-6837b292cd9b None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:53.650123 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e4ace8-aa6a-4449-8a65-6837b292cd9b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:53.650572 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e4ace8-aa6a-4449-8a65-6837b292cd9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:53.651940 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e4ace8-aa6a-4449-8a65-6837b292cd9b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:53.660397 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 386/784] 10.222.0.27 () {56 vars in 1124 bytes} [Sun Sep 4 00:48:53 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:53.667254 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4ace8-aa6a-4449-8a65-6837b292cd9b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:53.689274 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:53.695949 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:53.729472 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ece07b-a72c-4a3e-8539-0f44c6fb8643 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:53.803544 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac4d87-7a43-42a7-b32c-86de99af1cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:53.804600 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac4d87-7a43-42a7-b32c-86de99af1cd3 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:53.805117 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac4d87-7a43-42a7-b32c-86de99af1cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:53.805593 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac4d87-7a43-42a7-b32c-86de99af1cd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:53.824474 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac4d87-7a43-42a7-b32c-86de99af1cd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:53.826326 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac4d87-7a43-42a7-b32c-86de99af1cd3 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:53.848791 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4ace8-aa6a-4449-8a65-6837b292cd9b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:53.849015 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-38ac4d87-7a43-42a7-b32c-86de99af1cd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:53.849015 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:53.849015 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:53.849015 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:53.849015 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:53.849015 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:53.849015 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:53.849015 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:53.849015 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:53.849015 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:53.849015 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:53.849015 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:53.849015 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:53.849015 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:53.849015 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:53.849015 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:53.849892 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:53.849892 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:53.849892 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:53.849892 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:53.849892 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:53.849892 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:53.849892 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:53.849892 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:53.849892 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:53.849892 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:53.849892 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:53.849892 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:53.849892 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:53.849892 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:53.849892 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:53.849892 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:53.849892 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:53.850809 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:53.850809 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:53.850809 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:53.854685 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 399/785] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:53.855404 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac4d87-7a43-42a7-b32c-86de99af1cd3 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:53.898882 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211aa209-8eaa-4dd3-9be7-31616023b8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:53.900554 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211aa209-8eaa-4dd3-9be7-31616023b8fb None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:53.900987 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211aa209-8eaa-4dd3-9be7-31616023b8fb None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:53.901409 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211aa209-8eaa-4dd3-9be7-31616023b8fb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:53.902742 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211aa209-8eaa-4dd3-9be7-31616023b8fb None admin] RBAC: Adding query filter params (interface=public, service_id=dab83f46e6c34c9f9280d3944d88fd20, region_id=RegionOne) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:53.903214 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211aa209-8eaa-4dd3-9be7-31616023b8fb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:53.905462 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211aa209-8eaa-4dd3-9be7-31616023b8fb None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:53.913056 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 387/786] 10.222.0.27 () {56 vars in 1251 bytes} [Sun Sep 4 00:48:53 2022] GET /identity/v3/endpoints?service_id=dab83f46e6c34c9f9280d3944d88fd20&interface=public®ion_id=RegionOne => generated 194 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:48:53.914977 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211aa209-8eaa-4dd3-9be7-31616023b8fb None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:53.966806 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac4d87-7a43-42a7-b32c-86de99af1cd3 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:53.995265 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:54.009561 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db652dc0-7967-42db-8749-8336ea419781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:54.010159 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db652dc0-7967-42db-8749-8336ea419781 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:54.010424 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db652dc0-7967-42db-8749-8336ea419781 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:54.010691 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db652dc0-7967-42db-8749-8336ea419781 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:54.023884 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db652dc0-7967-42db-8749-8336ea419781 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:54.027476 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:54.028739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-db652dc0-7967-42db-8749-8336ea419781 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:48:54.028739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:54.028739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:54.028739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:54.028739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:54.028739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:54.028739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:54.028739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:54.028739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:54.028739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:54.028739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:54.028739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:54.028739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:54.028739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:54.028739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:54.028739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:54.029698 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:54.029698 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:54.029698 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:54.029698 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:54.029698 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:54.029698 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:54.029698 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:54.029698 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:54.029698 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:54.029698 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:54.029698 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:54.029698 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:54.029698 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:54.029698 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:54.029698 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:54.029698 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:54.030528 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:54.030528 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:54.030528 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:54.030528 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:54.030528 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:54.030528 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:54.030528 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:54.030528 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:54.030528 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:54.030528 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:54.030528 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:48:54.030528 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:54.030528 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 400/787] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:48:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:54.031207 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db652dc0-7967-42db-8749-8336ea419781 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:54.064797 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211aa209-8eaa-4dd3-9be7-31616023b8fb None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:54.102703 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b70a7-c3db-4fcf-becf-3419ac876268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:54.103306 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b70a7-c3db-4fcf-becf-3419ac876268 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:54.103630 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b70a7-c3db-4fcf-becf-3419ac876268 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:54.103905 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b70a7-c3db-4fcf-becf-3419ac876268 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:54.104565 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b70a7-c3db-4fcf-becf-3419ac876268 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:54.104867 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b70a7-c3db-4fcf-becf-3419ac876268 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:54.106273 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b70a7-c3db-4fcf-becf-3419ac876268 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:54.118915 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 388/788] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:54.119913 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b70a7-c3db-4fcf-becf-3419ac876268 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:54.170466 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db652dc0-7967-42db-8749-8336ea419781 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:54.216533 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32934794-5b13-42a0-850f-1233eebde335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:54.217664 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32934794-5b13-42a0-850f-1233eebde335 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:54.218102 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32934794-5b13-42a0-850f-1233eebde335 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:54.218551 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32934794-5b13-42a0-850f-1233eebde335 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:54.219307 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32934794-5b13-42a0-850f-1233eebde335 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:54.221247 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32934794-5b13-42a0-850f-1233eebde335 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:54.232494 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 401/789] 10.222.0.27 () {56 vars in 1087 bytes} [Sun Sep 4 00:48:54 2022] GET /identity/v3/services => generated 772 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:54.233402 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32934794-5b13-42a0-850f-1233eebde335 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:54.241386 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b70a7-c3db-4fcf-becf-3419ac876268 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:54.291079 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada2e216-353f-4082-99c0-d7a320396721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:54.292109 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2e216-353f-4082-99c0-d7a320396721 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:54.292634 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2e216-353f-4082-99c0-d7a320396721 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:54.293113 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:54.293436 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2e216-353f-4082-99c0-d7a320396721 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:54.294949 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada2e216-353f-4082-99c0-d7a320396721 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:54.295641 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada2e216-353f-4082-99c0-d7a320396721 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:54.297496 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada2e216-353f-4082-99c0-d7a320396721 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:54.301402 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:54.310171 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 389/790] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:48:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:54.316210 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada2e216-353f-4082-99c0-d7a320396721 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:54.394593 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32934794-5b13-42a0-850f-1233eebde335 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:54.450111 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630d555-1c20-4dcd-b32d-7f259cbceb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:54.451554 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630d555-1c20-4dcd-b32d-7f259cbceb8d None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:54.451554 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630d555-1c20-4dcd-b32d-7f259cbceb8d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:54.451871 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630d555-1c20-4dcd-b32d-7f259cbceb8d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:54.462598 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5630d555-1c20-4dcd-b32d-7f259cbceb8d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:54.464790 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5630d555-1c20-4dcd-b32d-7f259cbceb8d None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:54.466528 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada2e216-353f-4082-99c0-d7a320396721 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:54.479712 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-5630d555-1c20-4dcd-b32d-7f259cbceb8d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 04 00:48:54.479712 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:54.479712 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:54.479712 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:54.479712 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:54.479712 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:54.479712 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:54.479712 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:54.479712 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:54.479712 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:54.479712 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:54.479712 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:54.479712 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:48:54.479712 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:48:54.479712 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:48:54.479712 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:48:54.480732 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:54.480732 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:54.480732 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:48:54.480732 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:54.480732 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:48:54.480732 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:54.480732 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:54.480732 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:54.480732 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:54.480732 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:54.480732 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:54.480732 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:54.480732 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:54.480732 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:54.480732 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:54.480732 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:54.480732 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:54.480732 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:54.481682 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:54.481682 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:54.481682 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:48:54.481682 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:48:54.481682 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:48:54.481682 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:48:54.481682 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:48:54.481682 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:48:54.481682 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 04 00:48:54.481682 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:54.481682 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 402/791] 10.222.0.27 () {56 vars in 1117 bytes} [Sun Sep 4 00:48:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:54.488031 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630d555-1c20-4dcd-b32d-7f259cbceb8d None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:54.512497 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793545d-1c84-4a88-877f-ef8670156ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:54.513563 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793545d-1c84-4a88-877f-ef8670156ab5 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:54.514065 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793545d-1c84-4a88-877f-ef8670156ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:54.514588 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793545d-1c84-4a88-877f-ef8670156ab5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:54.526851 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5793545d-1c84-4a88-877f-ef8670156ab5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:54.528786 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5793545d-1c84-4a88-877f-ef8670156ab5 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:54.549068 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-5793545d-1c84-4a88-877f-ef8670156ab5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:54.549068 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:54.549068 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:54.549068 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:54.549068 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:54.549068 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:54.549068 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:54.549068 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:54.549068 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:54.549068 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:54.549068 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:54.549068 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:54.549068 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:54.549068 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:54.549068 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:54.549068 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:54.550052 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:54.550052 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:54.550052 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:54.550052 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:54.550052 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:54.550052 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:54.550052 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:54.550052 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:54.550052 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:54.550052 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:54.550052 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:54.550052 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:54.550052 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:54.550052 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:54.550052 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:54.550052 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:54.550052 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:54.550939 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:54.550939 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:54.550939 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:54.554273 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 390/792] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:54.560229 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793545d-1c84-4a88-877f-ef8670156ab5 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:54.614890 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:54.669638 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793545d-1c84-4a88-877f-ef8670156ab5 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:54.705995 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a286155c-bc54-439e-9547-3cb66078900b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:54.706513 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286155c-bc54-439e-9547-3cb66078900b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:54.706796 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286155c-bc54-439e-9547-3cb66078900b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:54.707067 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286155c-bc54-439e-9547-3cb66078900b None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:54.707709 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a286155c-bc54-439e-9547-3cb66078900b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:54.708006 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a286155c-bc54-439e-9547-3cb66078900b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:54.709391 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a286155c-bc54-439e-9547-3cb66078900b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:54.709745 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630d555-1c20-4dcd-b32d-7f259cbceb8d None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:54.722570 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 391/793] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:54.728381 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a286155c-bc54-439e-9547-3cb66078900b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:54.736787 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:54.778589 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364ae57-4b81-4730-bc34-ad4921ed0d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:54.779200 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364ae57-4b81-4730-bc34-ad4921ed0d06 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:54.779478 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364ae57-4b81-4730-bc34-ad4921ed0d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:54.779753 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364ae57-4b81-4730-bc34-ad4921ed0d06 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:54.780488 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364ae57-4b81-4730-bc34-ad4921ed0d06 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:54.780800 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364ae57-4b81-4730-bc34-ad4921ed0d06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:54.783035 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364ae57-4b81-4730-bc34-ad4921ed0d06 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:54.820096 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 403/794] 10.222.0.27 () {56 vars in 1149 bytes} [Sun Sep 4 00:48:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:54.825453 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364ae57-4b81-4730-bc34-ad4921ed0d06 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:54.920714 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a286155c-bc54-439e-9547-3cb66078900b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:54.944853 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:54.980533 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364ae57-4b81-4730-bc34-ad4921ed0d06 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:54.990114 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166e2ad-1440-4669-bb2b-91ee5b4e48a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:54.991043 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166e2ad-1440-4669-bb2b-91ee5b4e48a8 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:54.991043 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166e2ad-1440-4669-bb2b-91ee5b4e48a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:54.991312 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166e2ad-1440-4669-bb2b-91ee5b4e48a8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:55.001643 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1166e2ad-1440-4669-bb2b-91ee5b4e48a8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:55.003717 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1166e2ad-1440-4669-bb2b-91ee5b4e48a8 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:55.019152 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-1166e2ad-1440-4669-bb2b-91ee5b4e48a8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 04 00:48:55.019152 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:55.019152 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:55.019152 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:55.019152 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:55.019152 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:55.019152 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:55.019152 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:55.019152 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:55.019152 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:55.019152 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:55.019152 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:55.019152 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:48:55.019152 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:48:55.019152 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:48:55.019152 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:48:55.020200 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:55.020200 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:55.020200 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:48:55.020200 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:55.020200 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:48:55.020200 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:55.020200 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:55.020200 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:55.020200 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:55.020200 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:55.020200 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:55.020200 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:55.020200 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:55.020200 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:55.020200 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:55.020200 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:55.020200 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:55.020200 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:55.021208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:55.021208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:55.021208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:48:55.021208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:48:55.021208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:48:55.021208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:48:55.021208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:48:55.021208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:48:55.021208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 04 00:48:55.021208 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:55.021208 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 392/795] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:48:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:55.021803 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2b314-89c1-4359-b642-78aba372daf9 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:55.022134 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2b314-89c1-4359-b642-78aba372daf9 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:55.022532 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2b314-89c1-4359-b642-78aba372daf9 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:55.024262 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 393/796] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:55.026438 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2b314-89c1-4359-b642-78aba372daf9 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:55.030465 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1032d13a-2e4c-44a9-bf97-59e36e1a4091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:55.031161 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1032d13a-2e4c-44a9-bf97-59e36e1a4091 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:55.031444 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1032d13a-2e4c-44a9-bf97-59e36e1a4091 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:55.031769 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1032d13a-2e4c-44a9-bf97-59e36e1a4091 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:55.044656 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1032d13a-2e4c-44a9-bf97-59e36e1a4091 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:55.046373 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1032d13a-2e4c-44a9-bf97-59e36e1a4091 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:55.062729 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-1032d13a-2e4c-44a9-bf97-59e36e1a4091 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:55.062729 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:55.062729 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:55.062729 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:55.062729 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:55.062729 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:55.062729 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:55.062729 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:55.062729 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:55.062729 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:55.062729 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:55.062729 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:55.062729 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:55.062729 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:55.062729 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:55.062729 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:55.064417 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:55.064417 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:55.064417 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:55.064417 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:55.064417 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:55.064417 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:55.064417 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:55.064417 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:55.064417 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:55.064417 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:55.064417 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:55.064417 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:55.064417 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:55.064417 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:55.064417 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:55.064417 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:55.064417 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:55.065396 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:55.065396 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:55.065396 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:55.068812 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 404/797] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:55.070424 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561eb06-0cf1-4e3a-b119-ba51c74af048 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:55.070714 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561eb06-0cf1-4e3a-b119-ba51c74af048 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:55.070994 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561eb06-0cf1-4e3a-b119-ba51c74af048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:55.091085 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-0561eb06-0cf1-4e3a-b119-ba51c74af048 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:55.092294 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-0561eb06-0cf1-4e3a-b119-ba51c74af048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:55.110818 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:55.148064 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2b314-89c1-4359-b642-78aba372daf9 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:55.169562 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 405/798] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:48:55.170592 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561eb06-0cf1-4e3a-b119-ba51c74af048 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:55.205192 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcac8b4-198b-4eac-82d3-0135a12e5132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:55.205810 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcac8b4-198b-4eac-82d3-0135a12e5132 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:55.206097 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcac8b4-198b-4eac-82d3-0135a12e5132 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:55.206372 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcac8b4-198b-4eac-82d3-0135a12e5132 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:55.207173 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcac8b4-198b-4eac-82d3-0135a12e5132 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:55.207486 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcac8b4-198b-4eac-82d3-0135a12e5132 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:55.208943 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcac8b4-198b-4eac-82d3-0135a12e5132 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:55.239996 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 394/799] 10.222.0.27 () {56 vars in 1135 bytes} [Sun Sep 4 00:48:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:55.244520 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcac8b4-198b-4eac-82d3-0135a12e5132 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:55.279027 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561eb06-0cf1-4e3a-b119-ba51c74af048 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:55.293380 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:55.318171 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6109010f-af89-41cc-b834-58cc2abfcb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:55.318786 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109010f-af89-41cc-b834-58cc2abfcb71 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:55.319068 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109010f-af89-41cc-b834-58cc2abfcb71 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:55.319344 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109010f-af89-41cc-b834-58cc2abfcb71 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:55.319986 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6109010f-af89-41cc-b834-58cc2abfcb71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:55.320292 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6109010f-af89-41cc-b834-58cc2abfcb71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:55.321807 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6109010f-af89-41cc-b834-58cc2abfcb71 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:55.334099 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 406/800] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:55.339622 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6109010f-af89-41cc-b834-58cc2abfcb71 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:55.368755 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:55.417434 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcac8b4-198b-4eac-82d3-0135a12e5132 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:55.455528 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d037d-d455-4132-bb92-c0345ca10ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:55.456268 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d037d-d455-4132-bb92-c0345ca10ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:55.456719 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d037d-d455-4132-bb92-c0345ca10ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:55.456994 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d037d-d455-4132-bb92-c0345ca10ba9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:55.467732 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d037d-d455-4132-bb92-c0345ca10ba9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:55.469444 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d037d-d455-4132-bb92-c0345ca10ba9 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:55.485706 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-d01d037d-d455-4132-bb92-c0345ca10ba9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:55.485706 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:55.485706 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:55.485706 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:55.485706 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:55.485706 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:55.485706 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:55.485706 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:55.485706 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:55.485706 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:55.485706 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:55.485706 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:55.485706 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:55.485706 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:55.485706 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:55.485706 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:55.486689 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:55.486689 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:55.486689 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:55.486689 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:55.486689 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:55.486689 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:55.486689 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:55.486689 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:55.486689 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:55.486689 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:55.486689 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:55.486689 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:55.486689 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:55.486689 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:55.486689 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:55.486689 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:55.486689 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:55.487769 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:55.487769 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:55.487769 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:55.487769 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 395/801] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:48:55.489322 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1ffd9-ae42-4483-8efa-44f7d8fca804 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:55.489646 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1ffd9-ae42-4483-8efa-44f7d8fca804 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:55.490014 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1ffd9-ae42-4483-8efa-44f7d8fca804 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:55.495138 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 396/802] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:55.495138 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b1ffd9-ae42-4483-8efa-44f7d8fca804 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:55.502844 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6109010f-af89-41cc-b834-58cc2abfcb71 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:55.539592 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4180cedb-bd07-41a0-b50a-32a658e8ca8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:55.540425 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180cedb-bd07-41a0-b50a-32a658e8ca8f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:55.540775 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180cedb-bd07-41a0-b50a-32a658e8ca8f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:55.541155 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180cedb-bd07-41a0-b50a-32a658e8ca8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:55.549790 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4180cedb-bd07-41a0-b50a-32a658e8ca8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:55.551532 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4180cedb-bd07-41a0-b50a-32a658e8ca8f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:55.565432 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-4180cedb-bd07-41a0-b50a-32a658e8ca8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:55.565432 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:55.565432 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:55.565432 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:55.565432 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:55.565432 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:55.565432 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:55.565432 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:55.565432 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:55.565432 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:55.565432 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:55.565432 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:55.565432 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:48:55.565432 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:48:55.565432 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:48:55.566409 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:48:55.566409 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:55.566409 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:55.566409 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:55.566409 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:55.566409 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:55.566409 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:55.566409 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:55.566409 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:55.566409 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:55.566409 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:55.566409 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:55.566409 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:55.566409 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:55.566409 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:55.566409 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:55.566409 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:55.567332 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:48:55.567332 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:48:55.567332 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:48:55.567332 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:48:55.567332 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:48:55.567332 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:48:55.567332 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:55.567332 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:55.567332 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 407/803] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:48:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:48:55.569093 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470eaa4-7fd8-4bd2-b649-2bb66db3df9f None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:55.569093 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470eaa4-7fd8-4bd2-b649-2bb66db3df9f None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:55.569343 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470eaa4-7fd8-4bd2-b649-2bb66db3df9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:55.587922 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-0470eaa4-7fd8-4bd2-b649-2bb66db3df9f None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:55.589152 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-0470eaa4-7fd8-4bd2-b649-2bb66db3df9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:55.599333 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:55.621792 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b1ffd9-ae42-4483-8efa-44f7d8fca804 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:55.659107 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e06d316-ae49-4d64-a53b-7e3857f249e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:55.659711 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06d316-ae49-4d64-a53b-7e3857f249e6 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:55.659996 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06d316-ae49-4d64-a53b-7e3857f249e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:55.660272 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06d316-ae49-4d64-a53b-7e3857f249e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:55.660926 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e06d316-ae49-4d64-a53b-7e3857f249e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:55.661223 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e06d316-ae49-4d64-a53b-7e3857f249e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:55.662581 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e06d316-ae49-4d64-a53b-7e3857f249e6 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:55.667485 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 408/804] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:48:55.668716 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0470eaa4-7fd8-4bd2-b649-2bb66db3df9f None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:55.682082 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:55.684818 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 397/805] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:48:55.685665 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e06d316-ae49-4d64-a53b-7e3857f249e6 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:55.796743 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e06d316-ae49-4d64-a53b-7e3857f249e6 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:55.801103 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0470eaa4-7fd8-4bd2-b649-2bb66db3df9f None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:55.837527 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541ab8fa-0721-41d0-a79b-4492bc1ebbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:55.838189 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ab8fa-0721-41d0-a79b-4492bc1ebbac None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:55.838470 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ab8fa-0721-41d0-a79b-4492bc1ebbac None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:55.838798 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ab8fa-0721-41d0-a79b-4492bc1ebbac None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:55.839493 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541ab8fa-0721-41d0-a79b-4492bc1ebbac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:55.839798 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541ab8fa-0721-41d0-a79b-4492bc1ebbac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:55.840975 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c30d7b-7117-48c2-b870-363242fee293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:55.841256 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541ab8fa-0721-41d0-a79b-4492bc1ebbac None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:55.841606 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c30d7b-7117-48c2-b870-363242fee293 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:55.841887 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c30d7b-7117-48c2-b870-363242fee293 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:55.842162 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c30d7b-7117-48c2-b870-363242fee293 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:55.852575 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:55.857291 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c30d7b-7117-48c2-b870-363242fee293 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:55.862957 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 398/806] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:48:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:55.865414 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-a4c30d7b-7117-48c2-b870-363242fee293 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:48:55.865414 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:55.865414 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:55.865414 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:55.865414 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:55.865414 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:55.865414 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:55.865414 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:55.865414 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:55.865414 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:55.865414 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:55.865414 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:55.865414 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:55.865414 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:55.865414 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:55.865414 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:55.866423 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:55.866423 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:55.866423 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:55.866423 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:55.866423 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:55.866423 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:55.866423 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:55.866423 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:55.866423 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:55.866423 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:55.866423 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:55.866423 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:55.866423 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:55.866423 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:55.866423 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:55.866423 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:55.867290 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:55.867290 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:55.867290 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:55.867290 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:55.867290 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:55.867290 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:55.867290 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:55.867290 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:55.867290 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:55.867290 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:55.867290 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:48:55.867290 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:55.867922 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541ab8fa-0721-41d0-a79b-4492bc1ebbac None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:55.867996 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 409/807] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:48:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:55.869128 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c30d7b-7117-48c2-b870-363242fee293 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:55.988759 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541ab8fa-0721-41d0-a79b-4492bc1ebbac None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:56.027084 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2219d82d-7eed-47c7-b83b-7f7c7bf44230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:56.027720 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2219d82d-7eed-47c7-b83b-7f7c7bf44230 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:56.028121 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2219d82d-7eed-47c7-b83b-7f7c7bf44230 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:56.028445 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2219d82d-7eed-47c7-b83b-7f7c7bf44230 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:56.035849 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2219d82d-7eed-47c7-b83b-7f7c7bf44230 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:56.037501 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2219d82d-7eed-47c7-b83b-7f7c7bf44230 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:56.050353 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-2219d82d-7eed-47c7-b83b-7f7c7bf44230 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:56.050353 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:56.050353 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:56.050353 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:56.050353 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:56.050353 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:56.050353 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:56.050353 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:56.050353 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:56.050353 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:56.050353 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:56.050353 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:56.050353 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:48:56.050353 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:48:56.050353 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:48:56.051240 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:48:56.051240 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:56.051240 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:56.051240 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:56.051240 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:56.051240 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:56.051240 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:56.051240 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:56.051240 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:56.051240 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:56.051240 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:56.051240 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:56.051240 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:56.051240 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:56.051240 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:56.051240 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:56.051240 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:56.052097 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:48:56.052097 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:48:56.052097 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:48:56.052097 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:48:56.052097 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:48:56.052097 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:48:56.052097 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:56.052097 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:56.052097 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 399/808] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:48:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:56.053266 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2219d82d-7eed-47c7-b83b-7f7c7bf44230 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:56.072242 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:56.115062 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c30d7b-7117-48c2-b870-363242fee293 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:56.115726 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:56.151173 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30964f-3d4c-4aa8-8516-87cc05b66f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:56.151842 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30964f-3d4c-4aa8-8516-87cc05b66f5c None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:56.152130 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30964f-3d4c-4aa8-8516-87cc05b66f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:56.152410 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30964f-3d4c-4aa8-8516-87cc05b66f5c None admin] PATH_INFO: `/v3/projects/5100a8d316d243fb8f82e9044e1b3b06/users/d6d01ca673a1404c9b4cea51036c595c/roles/6e3e398ba29e436e8532a45d2fdf1321` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:56.160022 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2219d82d-7eed-47c7-b83b-7f7c7bf44230 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:56.168269 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed30964f-3d4c-4aa8-8516-87cc05b66f5c None admin] RBAC: Authorizing `identity:create_grant(project_id=5100a8d316d243fb8f82e9044e1b3b06, user_id=d6d01ca673a1404c9b4cea51036c595c, role_id=6e3e398ba29e436e8532a45d2fdf1321)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:56.169773 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed30964f-3d4c-4aa8-8516-87cc05b66f5c None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:56.195306 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae52368-595e-4039-8918-0307da64cdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:56.195932 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae52368-595e-4039-8918-0307da64cdb3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:56.196214 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae52368-595e-4039-8918-0307da64cdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:56.196509 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae52368-595e-4039-8918-0307da64cdb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:56.197165 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae52368-595e-4039-8918-0307da64cdb3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:56.197464 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae52368-595e-4039-8918-0307da64cdb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:56.198801 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae52368-595e-4039-8918-0307da64cdb3 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:56.206760 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 400/809] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:48:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:56.207531 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae52368-595e-4039-8918-0307da64cdb3 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:56.256628 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 410/810] 10.222.0.27 () {58 vars in 1439 bytes} [Sun Sep 4 00:48:55 2022] PUT /identity/v3/projects/5100a8d316d243fb8f82e9044e1b3b06/users/d6d01ca673a1404c9b4cea51036c595c/roles/6e3e398ba29e436e8532a45d2fdf1321 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 00:48:56.257836 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30964f-3d4c-4aa8-8516-87cc05b66f5c None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:56.354147 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae52368-595e-4039-8918-0307da64cdb3 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:56.389999 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:56.391138 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30964f-3d4c-4aa8-8516-87cc05b66f5c None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:56.403995 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4587d53-4b27-441c-b5d3-92bd20b2f965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:56.404827 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4587d53-4b27-441c-b5d3-92bd20b2f965 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:56.406551 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4587d53-4b27-441c-b5d3-92bd20b2f965 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:56.406551 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4587d53-4b27-441c-b5d3-92bd20b2f965 None admin] PATH_INFO: `/v3/services/network` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:56.406551 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4587d53-4b27-441c-b5d3-92bd20b2f965 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:56.408236 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4587d53-4b27-441c-b5d3-92bd20b2f965 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:56.410085 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:56.420352 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-e4587d53-4b27-441c-b5d3-92bd20b2f965 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 04 00:48:56.420352 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:56.420352 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 00:48:56.420352 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 00:48:56.420352 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 00:48:56.420352 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 00:48:56.420352 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 00:48:56.420352 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:48:56.420352 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 04 00:48:56.420352 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:56.420352 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 00:48:56.420352 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:56.420352 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:56.420352 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:56.420352 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:56.420352 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:56.420352 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:56.421388 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:56.421388 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:56.421388 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:56.421388 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:56.421388 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:56.421388 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:56.421388 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 00:48:56.421388 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 00:48:56.421388 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 00:48:56.421388 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 00:48:56.421388 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:56.421388 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:56.421388 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:56.421388 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:56.421388 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:56.421388 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:56.423188 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:56.423188 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:56.423188 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:56.423188 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:56.423188 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:56.423188 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:56.423188 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:56.423188 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:56.423188 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:56.423188 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:56.423188 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 00:48:56.423188 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:48:56.423188 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 04 00:48:56.423188 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:56.423188 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 401/811] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:48:56 2022] GET /identity/v3/services/network => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:56.423994 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4587d53-4b27-441c-b5d3-92bd20b2f965 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:56.436386 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382c4a0-6046-480d-bded-7e9cfb92131a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:56.437033 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382c4a0-6046-480d-bded-7e9cfb92131a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:56.437413 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382c4a0-6046-480d-bded-7e9cfb92131a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:56.437666 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382c4a0-6046-480d-bded-7e9cfb92131a None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:56.438591 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4382c4a0-6046-480d-bded-7e9cfb92131a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:56.438591 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4382c4a0-6046-480d-bded-7e9cfb92131a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:56.440340 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4382c4a0-6046-480d-bded-7e9cfb92131a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:56.452985 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 411/812] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:48:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:56.454327 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c15ec8-5f9e-4652-b480-f72106891fe6 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:56.454615 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c15ec8-5f9e-4652-b480-f72106891fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:56.454895 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c15ec8-5f9e-4652-b480-f72106891fe6 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:56.456243 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 412/813] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:56.461852 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c15ec8-5f9e-4652-b480-f72106891fe6 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:56.543637 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4587d53-4b27-441c-b5d3-92bd20b2f965 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:56.570140 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c15ec8-5f9e-4652-b480-f72106891fe6 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:56.580440 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1b2d8-c7d2-43a3-b3f4-cbdf25197118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:56.581426 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1b2d8-c7d2-43a3-b3f4-cbdf25197118 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:56.581854 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1b2d8-c7d2-43a3-b3f4-cbdf25197118 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:56.582274 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1b2d8-c7d2-43a3-b3f4-cbdf25197118 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:56.593304 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1b2d8-c7d2-43a3-b3f4-cbdf25197118 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:56.595256 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1b2d8-c7d2-43a3-b3f4-cbdf25197118 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:56.608805 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c5bb5-c82d-4f99-a084-3cf7fdd4b2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:56.609343 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c5bb5-c82d-4f99-a084-3cf7fdd4b2a9 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:56.609622 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c5bb5-c82d-4f99-a084-3cf7fdd4b2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:56.609891 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c5bb5-c82d-4f99-a084-3cf7fdd4b2a9 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:56.611059 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-3ff1b2d8-c7d2-43a3-b3f4-cbdf25197118 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:56.611059 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:56.611059 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:56.611059 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:56.611059 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:56.611059 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:56.611059 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:56.611059 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:56.611059 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:56.611059 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:56.611059 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:56.611059 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:56.611059 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:56.611059 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:56.611059 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:56.611059 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:56.611966 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:56.611966 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:56.611966 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:56.611966 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:56.611966 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:56.611966 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:56.611966 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:56.611966 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:56.611966 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:56.611966 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:56.611966 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:56.611966 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:56.611966 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:56.611966 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:56.611966 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:56.611966 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:56.611966 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:56.617621 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:56.617621 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:56.617621 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:56.617796 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713c5bb5-c82d-4f99-a084-3cf7fdd4b2a9 None admin] RBAC: Adding query filter params (name=network) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:56.617796 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713c5bb5-c82d-4f99-a084-3cf7fdd4b2a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:56.617932 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 402/814] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:56.618004 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713c5bb5-c82d-4f99-a084-3cf7fdd4b2a9 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:56.618072 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99abe3a-bd75-4f30-9592-ceed07ad427a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:56.618072 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99abe3a-bd75-4f30-9592-ceed07ad427a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:56.618072 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99abe3a-bd75-4f30-9592-ceed07ad427a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:56.618264 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 413/815] 10.222.0.27 () {56 vars in 1112 bytes} [Sun Sep 4 00:48:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:56.618341 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c5bb5-c82d-4f99-a084-3cf7fdd4b2a9 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:56.638741 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-e99abe3a-bd75-4f30-9592-ceed07ad427a None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:56.639915 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e99abe3a-bd75-4f30-9592-ceed07ad427a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:56.654830 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:56.672222 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:56.720501 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 403/816] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:48:56.720501 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99abe3a-bd75-4f30-9592-ceed07ad427a None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:56.741927 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c5bb5-c82d-4f99-a084-3cf7fdd4b2a9 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:56.780718 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bce6c-7f33-4a6b-a835-102260baded4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:56.781463 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bce6c-7f33-4a6b-a835-102260baded4 None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:56.781805 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bce6c-7f33-4a6b-a835-102260baded4 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:56.782754 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bce6c-7f33-4a6b-a835-102260baded4 None admin] PATH_INFO: `/v3/projects/5100a8d316d243fb8f82e9044e1b3b06/users/2297d8b3fe8e47eb9fd39a6e3eea8f54/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:56.785021 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73bce6c-7f33-4a6b-a835-102260baded4 None admin] RBAC: Authorizing `identity:create_grant(project_id=5100a8d316d243fb8f82e9044e1b3b06, user_id=2297d8b3fe8e47eb9fd39a6e3eea8f54, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:56.786612 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73bce6c-7f33-4a6b-a835-102260baded4 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:56.832372 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99abe3a-bd75-4f30-9592-ceed07ad427a None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:56.851008 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 414/817] 10.222.0.27 () {58 vars in 1439 bytes} [Sun Sep 4 00:48:56 2022] PUT /identity/v3/projects/5100a8d316d243fb8f82e9044e1b3b06/users/2297d8b3fe8e47eb9fd39a6e3eea8f54/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 00:48:56.856751 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bce6c-7f33-4a6b-a835-102260baded4 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:56.874259 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410e858-7cc7-4955-851c-f8c03555bcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:56.875462 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410e858-7cc7-4955-851c-f8c03555bcb7 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:56.876773 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410e858-7cc7-4955-851c-f8c03555bcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:56.877551 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410e858-7cc7-4955-851c-f8c03555bcb7 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:56.878579 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6410e858-7cc7-4955-851c-f8c03555bcb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:56.879126 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6410e858-7cc7-4955-851c-f8c03555bcb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:56.881172 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6410e858-7cc7-4955-851c-f8c03555bcb7 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:56.904720 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 404/818] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:56.911425 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410e858-7cc7-4955-851c-f8c03555bcb7 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:56.988327 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:57.012744 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:57.020282 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bce6c-7f33-4a6b-a835-102260baded4 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:57.067535 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89591fcb-3b3d-4018-aaef-d0fd8e368b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:57.068743 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89591fcb-3b3d-4018-aaef-d0fd8e368b56 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:57.070159 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89591fcb-3b3d-4018-aaef-d0fd8e368b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:57.070595 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89591fcb-3b3d-4018-aaef-d0fd8e368b56 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:57.071387 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89591fcb-3b3d-4018-aaef-d0fd8e368b56 None admin] RBAC: Adding query filter params (type=network) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:57.071944 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89591fcb-3b3d-4018-aaef-d0fd8e368b56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:57.073535 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89591fcb-3b3d-4018-aaef-d0fd8e368b56 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:57.082206 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 415/819] 10.222.0.27 () {56 vars in 1112 bytes} [Sun Sep 4 00:48:56 2022] GET /identity/v3/services?type=network => generated 350 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:57.083053 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89591fcb-3b3d-4018-aaef-d0fd8e368b56 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:57.110011 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410e858-7cc7-4955-851c-f8c03555bcb7 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:57.154075 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5445578a-0dbf-40a2-acca-8c40ba9db7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:57.154645 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5445578a-0dbf-40a2-acca-8c40ba9db7f4 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:57.155056 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5445578a-0dbf-40a2-acca-8c40ba9db7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:57.155459 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5445578a-0dbf-40a2-acca-8c40ba9db7f4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:57.164414 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5445578a-0dbf-40a2-acca-8c40ba9db7f4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:57.166066 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5445578a-0dbf-40a2-acca-8c40ba9db7f4 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:57.180913 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-5445578a-0dbf-40a2-acca-8c40ba9db7f4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 04 00:48:57.180913 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:57.180913 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:57.180913 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:57.180913 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:57.180913 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:57.180913 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:57.180913 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:57.180913 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:57.180913 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:57.180913 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:57.180913 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:57.180913 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:48:57.180913 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:48:57.180913 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:48:57.180913 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:48:57.181980 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:57.181980 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:57.181980 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:48:57.181980 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:57.181980 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:48:57.181980 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:57.181980 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:57.181980 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:57.181980 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:57.181980 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:57.181980 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:57.181980 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:57.181980 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:57.181980 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:57.181980 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:57.181980 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:57.181980 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:57.181980 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:57.182918 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:57.182918 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:57.182918 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:48:57.182918 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:48:57.182918 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:48:57.182918 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:48:57.182918 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:48:57.182918 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:48:57.182918 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 04 00:48:57.182918 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:57.182918 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 405/820] 10.222.0.27 () {56 vars in 1093 bytes} [Sun Sep 4 00:48:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:48:57.183524 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5445578a-0dbf-40a2-acca-8c40ba9db7f4 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:57.217948 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89591fcb-3b3d-4018-aaef-d0fd8e368b56 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:57.258574 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a755c5-9db8-469a-8317-400099323ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:57.259290 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a755c5-9db8-469a-8317-400099323ec0 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:57.259566 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a755c5-9db8-469a-8317-400099323ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:57.259833 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a755c5-9db8-469a-8317-400099323ec0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:57.260409 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a755c5-9db8-469a-8317-400099323ec0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:57.261805 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a755c5-9db8-469a-8317-400099323ec0 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:57.271487 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-66a755c5-9db8-469a-8317-400099323ec0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 04 00:48:57.271487 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:57.271487 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 00:48:57.271487 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 00:48:57.271487 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 00:48:57.271487 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 00:48:57.271487 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 00:48:57.271487 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:48:57.271487 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 04 00:48:57.271487 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:57.271487 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 00:48:57.271487 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:57.271487 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:57.271487 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:57.271487 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:57.271487 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:57.271487 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:57.272497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:57.272497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:57.272497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:57.272497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:57.272497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:57.272497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:57.272497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 00:48:57.272497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 00:48:57.272497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 00:48:57.272497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 00:48:57.272497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:57.272497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:57.272497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:57.272497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:57.272497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:57.272497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:57.273314 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:57.273314 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:57.273314 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:57.273314 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:57.273314 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:57.273314 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:57.273314 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:57.273314 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:57.273314 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:57.273314 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:57.273314 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 00:48:57.273314 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:48:57.273314 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 04 00:48:57.273314 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:57.277413 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 416/821] 10.222.0.27 () {56 vars in 1129 bytes} [Sun Sep 4 00:48:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:57.278306 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a755c5-9db8-469a-8317-400099323ec0 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:57.293207 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:57.305939 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:57.311305 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5445578a-0dbf-40a2-acca-8c40ba9db7f4 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:57.358685 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bae23a-0412-4cc2-acd5-020861cf5ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:57.359412 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bae23a-0412-4cc2-acd5-020861cf5ad5 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:57.359791 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bae23a-0412-4cc2-acd5-020861cf5ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:57.362071 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bae23a-0412-4cc2-acd5-020861cf5ad5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:57.363550 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bae23a-0412-4cc2-acd5-020861cf5ad5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:57.365281 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bae23a-0412-4cc2-acd5-020861cf5ad5 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:57.386848 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a755c5-9db8-469a-8317-400099323ec0 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:57.427100 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea5f07b-86c6-4bed-8b0c-17b3ca462db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:57.427701 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5f07b-86c6-4bed-8b0c-17b3ca462db2 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:57.427984 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5f07b-86c6-4bed-8b0c-17b3ca462db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:57.428257 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5f07b-86c6-4bed-8b0c-17b3ca462db2 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:57.429015 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea5f07b-86c6-4bed-8b0c-17b3ca462db2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:57.429321 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea5f07b-86c6-4bed-8b0c-17b3ca462db2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:57.430746 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea5f07b-86c6-4bed-8b0c-17b3ca462db2 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:57.448366 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 417/822] 10.222.0.27 () {56 vars in 1133 bytes} [Sun Sep 4 00:48:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:57.449250 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea5f07b-86c6-4bed-8b0c-17b3ca462db2 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:57.485113 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 406/823] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 00:48:57 2022] POST /identity/v3/endpoints => generated 337 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:48:57.487298 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bae23a-0412-4cc2-acd5-020861cf5ad5 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:57.604802 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:57.626316 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea5f07b-86c6-4bed-8b0c-17b3ca462db2 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:57.680030 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3e902-2336-47e8-a28e-e9994c744ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:57.680609 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3e902-2336-47e8-a28e-e9994c744ea8 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:57.680887 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3e902-2336-47e8-a28e-e9994c744ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:57.681155 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3e902-2336-47e8-a28e-e9994c744ea8 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:57.681779 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b3e902-2336-47e8-a28e-e9994c744ea8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:57.682079 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b3e902-2336-47e8-a28e-e9994c744ea8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:57.683449 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b3e902-2336-47e8-a28e-e9994c744ea8 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:57.697372 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 418/824] 10.222.0.27 () {56 vars in 1124 bytes} [Sun Sep 4 00:48:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:57.699528 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3e902-2336-47e8-a28e-e9994c744ea8 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:57.707400 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bae23a-0412-4cc2-acd5-020861cf5ad5 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:57.753778 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a144fc-9c82-451b-9ea9-717d0875b57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:57.754917 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a144fc-9c82-451b-9ea9-717d0875b57a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:57.755374 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a144fc-9c82-451b-9ea9-717d0875b57a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:57.755816 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a144fc-9c82-451b-9ea9-717d0875b57a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:57.773128 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:57.841884 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a144fc-9c82-451b-9ea9-717d0875b57a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:57.845123 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a144fc-9c82-451b-9ea9-717d0875b57a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:57.846751 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3e902-2336-47e8-a28e-e9994c744ea8 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:57.885060 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b20b32-e0fa-4af2-8d22-297c60a75355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:57.885772 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b20b32-e0fa-4af2-8d22-297c60a75355 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:57.886115 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b20b32-e0fa-4af2-8d22-297c60a75355 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:57.886455 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b20b32-e0fa-4af2-8d22-297c60a75355 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:57.887147 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b20b32-e0fa-4af2-8d22-297c60a75355 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:57.887517 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b20b32-e0fa-4af2-8d22-297c60a75355 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:57.889149 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b20b32-e0fa-4af2-8d22-297c60a75355 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:57.899528 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 419/825] 10.222.0.27 () {56 vars in 1124 bytes} [Sun Sep 4 00:48:57 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:57.900783 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b20b32-e0fa-4af2-8d22-297c60a75355 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:57.910270 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:57.978833 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 407/826] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:48:57 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 04 00:48:57.979710 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a144fc-9c82-451b-9ea9-717d0875b57a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:58.011090 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b20b32-e0fa-4af2-8d22-297c60a75355 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:58.030013 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:58.047201 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7651698-e88d-4c73-b988-f32dcc5b57bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:58.048436 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7651698-e88d-4c73-b988-f32dcc5b57bf None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:58.048794 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7651698-e88d-4c73-b988-f32dcc5b57bf None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:58.049131 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7651698-e88d-4c73-b988-f32dcc5b57bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:58.049805 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7651698-e88d-4c73-b988-f32dcc5b57bf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:58.051249 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7651698-e88d-4c73-b988-f32dcc5b57bf None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:58.093315 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a144fc-9c82-451b-9ea9-717d0875b57a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:58.136983 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 420/827] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 00:48:57 2022] POST /identity/v3/endpoints => generated 346 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:48:58.147664 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13d696-a7a4-48b2-b6d5-de96ea72a486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:58.149199 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13d696-a7a4-48b2-b6d5-de96ea72a486 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:58.149733 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13d696-a7a4-48b2-b6d5-de96ea72a486 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:58.150210 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13d696-a7a4-48b2-b6d5-de96ea72a486 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:58.164042 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f13d696-a7a4-48b2-b6d5-de96ea72a486 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:58.166197 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f13d696-a7a4-48b2-b6d5-de96ea72a486 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:58.183478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-3f13d696-a7a4-48b2-b6d5-de96ea72a486 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:58.183478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:58.183478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:58.183478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:58.183478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:58.183478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:58.183478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:58.183478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:58.183478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:58.183478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:58.183478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:58.183478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:58.183478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:58.183478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:58.183478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:58.183478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:58.185818 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:58.185818 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:58.185818 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:58.185818 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:58.185818 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:58.185818 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:58.185818 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:58.185818 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:58.185818 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:58.185818 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:58.185818 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:58.185818 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:58.185818 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:58.185818 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:58.185818 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:58.185818 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:58.185818 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:58.186755 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:58.186755 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:58.186755 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:58.186755 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 408/828] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:58.194101 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7651698-e88d-4c73-b988-f32dcc5b57bf None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:58.270121 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:58.311464 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328eafe-d1b0-4a47-b80c-a40447801afd None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:58.311672 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328eafe-d1b0-4a47-b80c-a40447801afd None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:58.311892 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328eafe-d1b0-4a47-b80c-a40447801afd None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:58.314132 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 409/829] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:48:58.316232 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7651698-e88d-4c73-b988-f32dcc5b57bf None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:58.320237 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690e16b-2b8e-40ae-9cf5-c2a258fad6ce None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:58.320549 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690e16b-2b8e-40ae-9cf5-c2a258fad6ce None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:58.323551 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690e16b-2b8e-40ae-9cf5-c2a258fad6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:58.344614 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5690e16b-2b8e-40ae-9cf5-c2a258fad6ce None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:58.345630 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5690e16b-2b8e-40ae-9cf5-c2a258fad6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:58.362200 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b7c72-875f-44eb-bcfe-a9bd460f847d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:58.362823 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b7c72-875f-44eb-bcfe-a9bd460f847d None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:58.363103 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b7c72-875f-44eb-bcfe-a9bd460f847d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:58.363376 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b7c72-875f-44eb-bcfe-a9bd460f847d None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:58.364027 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b7c72-875f-44eb-bcfe-a9bd460f847d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:58.364333 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b7c72-875f-44eb-bcfe-a9bd460f847d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:58.365758 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b7c72-875f-44eb-bcfe-a9bd460f847d None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:58.379445 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 421/830] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:58.386861 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b7c72-875f-44eb-bcfe-a9bd460f847d None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:58.429607 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 410/831] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:58 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:48:58.508749 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b7c72-875f-44eb-bcfe-a9bd460f847d None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:58.533776 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:58.545801 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed053a3-6895-4ffa-87a1-ccf682fda7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:58.546434 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed053a3-6895-4ffa-87a1-ccf682fda7f7 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:58.552230 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed053a3-6895-4ffa-87a1-ccf682fda7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:58.552230 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed053a3-6895-4ffa-87a1-ccf682fda7f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:58.554257 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed053a3-6895-4ffa-87a1-ccf682fda7f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:58.555708 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed053a3-6895-4ffa-87a1-ccf682fda7f7 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:58.567913 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-4ed053a3-6895-4ffa-87a1-ccf682fda7f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:58.567913 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:58.567913 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:58.567913 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:58.567913 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:58.567913 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:58.567913 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:58.567913 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:58.567913 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:58.567913 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:58.567913 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:58.567913 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:58.567913 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:48:58.567913 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:48:58.567913 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:48:58.568860 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:48:58.568860 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:58.568860 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:58.568860 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:58.568860 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:58.568860 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:58.568860 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:58.568860 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:58.568860 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:58.568860 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:58.568860 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:58.568860 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:58.568860 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:58.568860 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:58.568860 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:58.568860 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:58.568860 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:58.569715 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:48:58.569715 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:48:58.569715 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:48:58.569715 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:48:58.569715 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:48:58.569715 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:48:58.569715 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:48:58.569715 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:58.569715 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 422/832] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:48:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:58.575285 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690e16b-2b8e-40ae-9cf5-c2a258fad6ce None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:58.602338 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:58.682538 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690e16b-2b8e-40ae-9cf5-c2a258fad6ce None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:58.710897 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed053a3-6895-4ffa-87a1-ccf682fda7f7 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:58.720769 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e38163-9d5f-49e3-a449-4156ad5585fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:58.721841 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e38163-9d5f-49e3-a449-4156ad5585fe None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:58.722126 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e38163-9d5f-49e3-a449-4156ad5585fe None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:58.722396 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e38163-9d5f-49e3-a449-4156ad5585fe None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:58.723094 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e38163-9d5f-49e3-a449-4156ad5585fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:58.723397 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e38163-9d5f-49e3-a449-4156ad5585fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:58.725286 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e38163-9d5f-49e3-a449-4156ad5585fe None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:58.741560 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 411/833] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:48:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:58.747236 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e38163-9d5f-49e3-a449-4156ad5585fe None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:58.840156 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:58.843245 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed053a3-6895-4ffa-87a1-ccf682fda7f7 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:58.859956 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e38163-9d5f-49e3-a449-4156ad5585fe None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:58.889757 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a0733-7a63-4db5-8ee5-48b6a3f6a8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:58.891111 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a0733-7a63-4db5-8ee5-48b6a3f6a8cd None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:58.892010 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a0733-7a63-4db5-8ee5-48b6a3f6a8cd None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:58.892481 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a0733-7a63-4db5-8ee5-48b6a3f6a8cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:58.902437 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c69443-6bd1-40aa-a231-d2ef7d7e894d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:58.903036 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c69443-6bd1-40aa-a231-d2ef7d7e894d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:58.903312 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c69443-6bd1-40aa-a231-d2ef7d7e894d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:58.903580 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c69443-6bd1-40aa-a231-d2ef7d7e894d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:58.904293 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c69443-6bd1-40aa-a231-d2ef7d7e894d None admin] RBAC: Adding query filter params (role.id=c8a59d34ee2f4cefa6940cac778260c7, scope.project.id=5100a8d316d243fb8f82e9044e1b3b06, user.id=68bc00bda5194cd8a7928f8c4df4ccd1) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:58.905184 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c69443-6bd1-40aa-a231-d2ef7d7e894d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:58.907984 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c69443-6bd1-40aa-a231-d2ef7d7e894d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:58.912377 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a0733-7a63-4db5-8ee5-48b6a3f6a8cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:58.918311 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-186a0733-7a63-4db5-8ee5-48b6a3f6a8cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:58.918311 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:58.918311 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:58.918311 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:58.918311 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:58.918311 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:58.918311 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:58.918311 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:58.918311 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:58.918311 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:58.918311 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:58.918311 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:58.918311 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:58.918311 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:58.918311 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:58.918311 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:58.919271 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:58.919271 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:58.919271 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:58.919271 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:58.919271 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:58.919271 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:58.919271 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:58.919271 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:58.919271 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:58.919271 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:58.919271 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:58.919271 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:58.919271 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:58.919271 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:58.919271 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:58.919271 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:58.920087 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:58.920087 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:58.920087 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:58.920087 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:58.920087 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:58.920087 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:58.920087 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:58.920087 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:58.920087 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:58.920087 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:58.920087 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:48:58.920087 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:58.921632 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 412/834] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:48:58 2022] GET /identity/v3/role_assignments?user.id=68bc00bda5194cd8a7928f8c4df4ccd1&scope.project.id=5100a8d316d243fb8f82e9044e1b3b06&role.id=c8a59d34ee2f4cefa6940cac778260c7 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:48:58.923336 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 423/835] 10.222.0.27 () {56 vars in 1102 bytes} [Sun Sep 4 00:48:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:48:58.929145 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c69443-6bd1-40aa-a231-d2ef7d7e894d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:58.977207 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874d0c8-409e-49cf-af63-fd932cefaaba None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:58.977862 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874d0c8-409e-49cf-af63-fd932cefaaba None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:58.978720 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874d0c8-409e-49cf-af63-fd932cefaaba None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:58.981345 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 424/836] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:48:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 00:48:58.987331 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6bc98-648c-4c52-9edd-7b79c0db53a6 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:58.987950 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6bc98-648c-4c52-9edd-7b79c0db53a6 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:58.988426 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6bc98-648c-4c52-9edd-7b79c0db53a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:59.008186 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-b5d6bc98-648c-4c52-9edd-7b79c0db53a6 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:48:59.009781 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b5d6bc98-648c-4c52-9edd-7b79c0db53a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:48:59.048495 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c69443-6bd1-40aa-a231-d2ef7d7e894d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:59.060231 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:59.088340 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453cf272-7a49-4f36-82d2-f939fe2f3416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:59.089429 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453cf272-7a49-4f36-82d2-f939fe2f3416 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:59.089943 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453cf272-7a49-4f36-82d2-f939fe2f3416 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:59.090467 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453cf272-7a49-4f36-82d2-f939fe2f3416 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:59.091346 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453cf272-7a49-4f36-82d2-f939fe2f3416 None admin] RBAC: Adding query filter params (name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:59.091794 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453cf272-7a49-4f36-82d2-f939fe2f3416 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:59.093323 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453cf272-7a49-4f36-82d2-f939fe2f3416 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:59.103667 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 425/837] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:48:58 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 00:48:59.105278 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 413/838] 10.222.0.27 () {56 vars in 1103 bytes} [Sun Sep 4 00:48:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:59.111801 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6bc98-648c-4c52-9edd-7b79c0db53a6 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:59.216904 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6bc98-648c-4c52-9edd-7b79c0db53a6 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:59.254862 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f5011-41ec-4bce-becf-8808f81a07b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:59.255504 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f5011-41ec-4bce-becf-8808f81a07b9 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:59.255786 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f5011-41ec-4bce-becf-8808f81a07b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:59.256064 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f5011-41ec-4bce-becf-8808f81a07b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:59.267383 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40f5011-41ec-4bce-becf-8808f81a07b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:59.268875 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40f5011-41ec-4bce-becf-8808f81a07b9 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:59.284841 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-f40f5011-41ec-4bce-becf-8808f81a07b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:59.284841 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:59.284841 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:59.284841 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:59.284841 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:59.284841 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:59.284841 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:59.284841 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:59.284841 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:59.284841 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:59.284841 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:59.284841 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:59.284841 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:48:59.284841 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:48:59.284841 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:48:59.284841 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:48:59.285784 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:59.285784 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:59.285784 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:59.285784 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:59.285784 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:59.285784 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:59.285784 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:59.285784 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:59.285784 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:59.285784 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:59.285784 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:59.285784 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:59.285784 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:59.285784 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:59.285784 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:59.285784 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:59.285784 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:48:59.286661 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:48:59.286661 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:48:59.286661 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:59.286661 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 426/839] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:59.293541 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453cf272-7a49-4f36-82d2-f939fe2f3416 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:59.313207 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:59.402828 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453cf272-7a49-4f36-82d2-f939fe2f3416 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:59.411874 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f5011-41ec-4bce-becf-8808f81a07b9 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:59.439951 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:59.441466 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152abcf-5689-4b33-8cbe-7fdafa8dd128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:59.442089 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152abcf-5689-4b33-8cbe-7fdafa8dd128 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:59.442377 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152abcf-5689-4b33-8cbe-7fdafa8dd128 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:59.442656 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152abcf-5689-4b33-8cbe-7fdafa8dd128 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:59.443289 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152abcf-5689-4b33-8cbe-7fdafa8dd128 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:59.443594 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152abcf-5689-4b33-8cbe-7fdafa8dd128 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:59.445015 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152abcf-5689-4b33-8cbe-7fdafa8dd128 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:59.457452 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 414/840] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:48:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:59.462785 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152abcf-5689-4b33-8cbe-7fdafa8dd128 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:59.536035 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f5011-41ec-4bce-becf-8808f81a07b9 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:59.578451 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:59.582912 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b6048a-30ba-4602-9495-0a3528f57866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:59.584293 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6048a-30ba-4602-9495-0a3528f57866 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:59.584752 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6048a-30ba-4602-9495-0a3528f57866 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:59.585172 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6048a-30ba-4602-9495-0a3528f57866 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:59.589741 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152abcf-5689-4b33-8cbe-7fdafa8dd128 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:59.598872 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b6048a-30ba-4602-9495-0a3528f57866 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:48:59.605692 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-45b6048a-30ba-4602-9495-0a3528f57866 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:48:59.605692 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:59.605692 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:59.605692 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:59.605692 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:59.605692 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:59.605692 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:59.605692 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:59.605692 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:59.605692 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:59.605692 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:59.605692 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:59.605692 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:48:59.605692 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:48:59.605692 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:48:59.605692 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:48:59.606823 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:48:59.606823 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:48:59.606823 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:48:59.606823 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:48:59.606823 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:59.606823 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:59.606823 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:59.606823 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:59.606823 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:59.606823 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:59.606823 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:59.606823 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:59.606823 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:59.606823 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:59.606823 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:59.606823 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:59.607937 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:59.607937 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:59.607937 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:59.607937 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:59.607937 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:48:59.607937 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:48:59.607937 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:48:59.607937 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:48:59.607937 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:48:59.607937 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:48:59.607937 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:48:59.607937 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:48:59.611534 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 427/841] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:48:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:48:59.617429 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b6048a-30ba-4602-9495-0a3528f57866 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:59.636994 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d57f8b-f30c-40a6-b63d-801b8a9f789b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:59.637671 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d57f8b-f30c-40a6-b63d-801b8a9f789b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:59.637951 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d57f8b-f30c-40a6-b63d-801b8a9f789b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:59.638279 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d57f8b-f30c-40a6-b63d-801b8a9f789b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:59.647814 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d57f8b-f30c-40a6-b63d-801b8a9f789b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:59.650007 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d57f8b-f30c-40a6-b63d-801b8a9f789b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:59.665492 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-c5d57f8b-f30c-40a6-b63d-801b8a9f789b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 04 00:48:59.665492 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:48:59.665492 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:48:59.665492 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:48:59.665492 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:48:59.665492 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:48:59.665492 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:48:59.665492 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:48:59.665492 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:48:59.665492 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:48:59.665492 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:48:59.665492 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:48:59.665492 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:48:59.665492 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:48:59.665492 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:48:59.665492 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:48:59.666478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:48:59.666478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:48:59.666478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:48:59.666478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:59.666478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:48:59.666478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:48:59.666478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:48:59.666478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:48:59.666478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:48:59.666478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:48:59.666478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:48:59.666478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:48:59.666478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:48:59.666478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:48:59.666478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:48:59.666478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:48:59.666478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:48:59.666478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:48:59.667412 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:48:59.667412 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:48:59.667412 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:48:59.667412 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:48:59.667412 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:48:59.667412 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:48:59.667412 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:48:59.667412 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:48:59.667412 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 04 00:48:59.667412 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:48:59.667412 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 415/842] 10.222.0.27 () {56 vars in 1117 bytes} [Sun Sep 4 00:48:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:48:59.673548 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d57f8b-f30c-40a6-b63d-801b8a9f789b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:59.730559 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:59.752876 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b6048a-30ba-4602-9495-0a3528f57866 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:59.796993 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42035e-9050-43b3-8bc0-37aa0553a06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:59.798121 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42035e-9050-43b3-8bc0-37aa0553a06e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:59.798395 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42035e-9050-43b3-8bc0-37aa0553a06e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:59.798666 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42035e-9050-43b3-8bc0-37aa0553a06e None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:59.799324 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b42035e-9050-43b3-8bc0-37aa0553a06e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:59.799624 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b42035e-9050-43b3-8bc0-37aa0553a06e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:59.801020 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b42035e-9050-43b3-8bc0-37aa0553a06e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:59.808532 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d57f8b-f30c-40a6-b63d-801b8a9f789b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:59.815078 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 428/843] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:48:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:48:59.815929 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42035e-9050-43b3-8bc0-37aa0553a06e None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:59.846330 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:48:59.855339 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054804f1-6d4f-4cf9-ba8c-ae0f7ba9d037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:48:59.856012 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054804f1-6d4f-4cf9-ba8c-ae0f7ba9d037 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:48:59.856282 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054804f1-6d4f-4cf9-ba8c-ae0f7ba9d037 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:48:59.856612 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054804f1-6d4f-4cf9-ba8c-ae0f7ba9d037 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:48:59.859931 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054804f1-6d4f-4cf9-ba8c-ae0f7ba9d037 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:48:59.859931 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054804f1-6d4f-4cf9-ba8c-ae0f7ba9d037 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:48:59.859931 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054804f1-6d4f-4cf9-ba8c-ae0f7ba9d037 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:48:59.878155 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 416/844] 10.222.0.27 () {56 vars in 1149 bytes} [Sun Sep 4 00:48:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:48:59.883232 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054804f1-6d4f-4cf9-ba8c-ae0f7ba9d037 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:48:59.962280 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42035e-9050-43b3-8bc0-37aa0553a06e None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:48:59.990191 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054804f1-6d4f-4cf9-ba8c-ae0f7ba9d037 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:00.008809 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa28247-7817-46bf-a9b4-35ee34922e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:00.009768 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa28247-7817-46bf-a9b4-35ee34922e4c None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:00.010064 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa28247-7817-46bf-a9b4-35ee34922e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:00.010346 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa28247-7817-46bf-a9b4-35ee34922e4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:00.027306 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3afad8-d65a-4371-8533-b42c5ebb2e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:00.027986 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3afad8-d65a-4371-8533-b42c5ebb2e7b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:00.028264 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3afad8-d65a-4371-8533-b42c5ebb2e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:00.028577 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3afad8-d65a-4371-8533-b42c5ebb2e7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:00.029788 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa28247-7817-46bf-a9b4-35ee34922e4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:00.031439 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa28247-7817-46bf-a9b4-35ee34922e4c None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:00.040595 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:00.051764 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-4fa28247-7817-46bf-a9b4-35ee34922e4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:49:00.051764 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:00.051764 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:00.051764 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:00.051764 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:00.051764 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:00.051764 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:00.051764 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:00.051764 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:00.051764 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:00.051764 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:00.051764 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:00.051764 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:49:00.051764 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:49:00.051764 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:49:00.051764 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:49:00.062644 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:00.062644 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:00.062644 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:00.062644 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:00.062644 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:00.062644 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:00.062644 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:00.062644 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:00.062644 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:00.062644 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:00.062644 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:00.062644 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:00.062644 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:00.062644 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:00.062644 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:00.062644 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:00.062644 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:49:00.063740 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:49:00.063740 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:49:00.063740 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:00.063740 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 429/845] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:48:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:00.063740 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa28247-7817-46bf-a9b4-35ee34922e4c None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:00.102447 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:00.117868 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3afad8-d65a-4371-8533-b42c5ebb2e7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:00.119733 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3afad8-d65a-4371-8533-b42c5ebb2e7b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:00.201245 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa28247-7817-46bf-a9b4-35ee34922e4c None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:00.241360 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 417/846] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:48:59 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 04 00:49:00.241360 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4dd61-afbd-414f-96eb-33282e17f374 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:00.241360 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4dd61-afbd-414f-96eb-33282e17f374 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:00.241360 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4dd61-afbd-414f-96eb-33282e17f374 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:00.241360 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 418/847] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:49:00.243369 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc4dd61-afbd-414f-96eb-33282e17f374 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:00.248224 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd7ad4-1944-4a39-beee-be5dfee7ef80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:00.249343 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd7ad4-1944-4a39-beee-be5dfee7ef80 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:00.249621 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd7ad4-1944-4a39-beee-be5dfee7ef80 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:00.249890 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd7ad4-1944-4a39-beee-be5dfee7ef80 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:00.251166 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd7ad4-1944-4a39-beee-be5dfee7ef80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:00.251613 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd7ad4-1944-4a39-beee-be5dfee7ef80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:00.253048 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd7ad4-1944-4a39-beee-be5dfee7ef80 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:00.265477 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 430/848] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:49:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:49:00.267316 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b8c57-c506-433e-866b-90bdc40d5fbf None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:00.267596 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b8c57-c506-433e-866b-90bdc40d5fbf None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:00.267867 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b8c57-c506-433e-866b-90bdc40d5fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:00.286436 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-c19b8c57-c506-433e-866b-90bdc40d5fbf None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:00.288543 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c19b8c57-c506-433e-866b-90bdc40d5fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:00.340375 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:00.349926 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:00.356495 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc4dd61-afbd-414f-96eb-33282e17f374 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:00.363751 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 431/849] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:00 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:49:00.364630 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b8c57-c506-433e-866b-90bdc40d5fbf None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:00.406185 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb858c-8082-4843-9d2f-8f9f56767d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:00.407495 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb858c-8082-4843-9d2f-8f9f56767d66 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:00.407868 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb858c-8082-4843-9d2f-8f9f56767d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:00.408189 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb858c-8082-4843-9d2f-8f9f56767d66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:00.423536 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb858c-8082-4843-9d2f-8f9f56767d66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:00.425462 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb858c-8082-4843-9d2f-8f9f56767d66 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:00.442932 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-60fb858c-8082-4843-9d2f-8f9f56767d66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:49:00.442932 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:00.442932 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:00.442932 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:00.442932 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:00.442932 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:00.442932 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:00.442932 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:00.442932 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:00.442932 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:00.442932 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:00.442932 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:00.442932 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:49:00.442932 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:49:00.442932 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:49:00.442932 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:49:00.444117 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:00.444117 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:00.444117 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:00.444117 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:00.444117 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:00.444117 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:00.444117 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:00.444117 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:00.444117 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:00.444117 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:00.444117 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:00.444117 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:00.444117 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:00.444117 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:00.444117 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:00.444117 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:00.444117 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:49:00.445289 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:49:00.445289 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:49:00.445289 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:00.451032 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 419/850] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:49:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:49:00.451286 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb858c-8082-4843-9d2f-8f9f56767d66 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:00.483836 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b8c57-c506-433e-866b-90bdc40d5fbf None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:00.525618 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526adf19-1451-4e02-8ef3-34fc1c6ac9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:00.526839 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526adf19-1451-4e02-8ef3-34fc1c6ac9f3 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:00.527569 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526adf19-1451-4e02-8ef3-34fc1c6ac9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:00.528482 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526adf19-1451-4e02-8ef3-34fc1c6ac9f3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:00.539314 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526adf19-1451-4e02-8ef3-34fc1c6ac9f3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:00.541936 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526adf19-1451-4e02-8ef3-34fc1c6ac9f3 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:00.558189 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-526adf19-1451-4e02-8ef3-34fc1c6ac9f3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 04 00:49:00.558189 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:00.558189 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:00.558189 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:00.558189 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:00.558189 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:00.558189 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:00.558189 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:00.558189 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:00.558189 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:00.558189 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:00.558189 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:00.558189 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:49:00.558189 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:49:00.558189 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:49:00.558189 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:49:00.559240 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:00.559240 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:00.559240 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:49:00.559240 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:49:00.559240 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:49:00.559240 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:49:00.559240 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:00.559240 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:00.559240 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:00.559240 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:00.559240 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:00.559240 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:00.559240 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:00.559240 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:00.559240 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:00.559240 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:00.559240 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:00.559240 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:00.560148 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:00.560148 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:00.560148 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:49:00.560148 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:49:00.560148 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:49:00.560148 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:49:00.560148 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:49:00.560148 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:49:00.560148 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 04 00:49:00.560148 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:00.563395 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 432/851] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:49:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:00.568554 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526adf19-1451-4e02-8ef3-34fc1c6ac9f3 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:00.584331 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb858c-8082-4843-9d2f-8f9f56767d66 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:00.635734 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26256273-9c29-451d-af54-8535ccd97d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:00.636859 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26256273-9c29-451d-af54-8535ccd97d6f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:00.637145 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26256273-9c29-451d-af54-8535ccd97d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:00.637683 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26256273-9c29-451d-af54-8535ccd97d6f None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:00.638370 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26256273-9c29-451d-af54-8535ccd97d6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:00.638740 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26256273-9c29-451d-af54-8535ccd97d6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:00.639217 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:00.642768 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26256273-9c29-451d-af54-8535ccd97d6f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:00.653639 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:00.665160 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 420/852] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:49:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:00.665160 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26256273-9c29-451d-af54-8535ccd97d6f None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:00.698331 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526adf19-1451-4e02-8ef3-34fc1c6ac9f3 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:00.739121 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdcb992-7d63-43ec-84de-1aff719769f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:00.739714 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdcb992-7d63-43ec-84de-1aff719769f9 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:00.740006 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdcb992-7d63-43ec-84de-1aff719769f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:00.740282 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdcb992-7d63-43ec-84de-1aff719769f9 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:00.741459 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdcb992-7d63-43ec-84de-1aff719769f9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:00.741774 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdcb992-7d63-43ec-84de-1aff719769f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:00.743600 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdcb992-7d63-43ec-84de-1aff719769f9 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:00.766799 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26256273-9c29-451d-af54-8535ccd97d6f None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:00.767699 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 433/853] 10.222.0.27 () {56 vars in 1135 bytes} [Sun Sep 4 00:49:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:49:00.768525 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdcb992-7d63-43ec-84de-1aff719769f9 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:00.803940 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d9ac3-5b46-4f15-a0cc-65fc137efa10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:00.804604 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d9ac3-5b46-4f15-a0cc-65fc137efa10 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:00.804881 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d9ac3-5b46-4f15-a0cc-65fc137efa10 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:00.805149 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d9ac3-5b46-4f15-a0cc-65fc137efa10 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:00.805738 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4d9ac3-5b46-4f15-a0cc-65fc137efa10 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:00.807076 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4d9ac3-5b46-4f15-a0cc-65fc137efa10 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:00.817215 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-5a4d9ac3-5b46-4f15-a0cc-65fc137efa10 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 04 00:49:00.817215 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:00.817215 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 00:49:00.817215 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 00:49:00.817215 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 00:49:00.817215 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 00:49:00.817215 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 00:49:00.817215 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:49:00.817215 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 04 00:49:00.817215 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:00.817215 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 00:49:00.817215 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:00.817215 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:00.817215 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:00.817215 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:00.817215 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:00.817215 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:00.818181 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:00.818181 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:00.818181 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:00.818181 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:00.818181 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:00.818181 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:00.818181 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 00:49:00.818181 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 00:49:00.818181 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 00:49:00.818181 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 00:49:00.818181 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:00.818181 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:00.818181 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:00.818181 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:00.818181 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:00.818181 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:00.819026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:00.819026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:00.819026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:00.819026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:00.819026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:00.819026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:00.819026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:00.819026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:00.819026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:00.819026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:00.819026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 00:49:00.819026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:49:00.819026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 04 00:49:00.819026 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:00.819026 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 421/854] 10.222.0.27 () {56 vars in 1114 bytes} [Sun Sep 4 00:49:00 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:00.819850 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d9ac3-5b46-4f15-a0cc-65fc137efa10 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:00.881653 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdcb992-7d63-43ec-84de-1aff719769f9 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:00.885300 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:00.890170 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:00.919200 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ce4e0-6521-4e2c-acd1-ed2835b2bcb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:00.919831 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ce4e0-6521-4e2c-acd1-ed2835b2bcb0 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:00.920111 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ce4e0-6521-4e2c-acd1-ed2835b2bcb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:00.920385 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ce4e0-6521-4e2c-acd1-ed2835b2bcb0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:00.927317 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675ce4e0-6521-4e2c-acd1-ed2835b2bcb0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:00.928859 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675ce4e0-6521-4e2c-acd1-ed2835b2bcb0 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:00.928973 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d9ac3-5b46-4f15-a0cc-65fc137efa10 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:00.942503 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-675ce4e0-6521-4e2c-acd1-ed2835b2bcb0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:49:00.942503 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:00.942503 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:00.942503 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:00.942503 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:00.942503 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:00.942503 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:00.942503 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:00.942503 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:00.942503 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:00.942503 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:00.942503 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:00.942503 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:49:00.942503 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:49:00.942503 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:49:00.943637 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:49:00.943637 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:00.943637 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:00.943637 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:00.943637 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:00.943637 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:00.943637 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:00.943637 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:00.943637 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:00.943637 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:00.943637 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:00.943637 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:00.943637 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:00.943637 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:00.943637 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:00.943637 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:00.943637 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:00.944562 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:49:00.944562 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:49:00.944562 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:49:00.944562 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:49:00.944562 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:49:00.944562 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:49:00.944562 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:49:00.944562 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:00.944562 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 434/855] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:49:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:00.945048 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ce4e0-6521-4e2c-acd1-ed2835b2bcb0 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:00.971285 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7ae3e-9830-4629-8035-804935292608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:00.971923 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7ae3e-9830-4629-8035-804935292608 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:00.972224 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7ae3e-9830-4629-8035-804935292608 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:00.972505 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7ae3e-9830-4629-8035-804935292608 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:01.040638 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b7ae3e-9830-4629-8035-804935292608 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:01.042075 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b7ae3e-9830-4629-8035-804935292608 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:01.044718 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ce4e0-6521-4e2c-acd1-ed2835b2bcb0 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:01.078004 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d40bf-33f1-4ebb-b22f-651d69625df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:01.078653 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d40bf-33f1-4ebb-b22f-651d69625df4 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:01.079424 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d40bf-33f1-4ebb-b22f-651d69625df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:01.079697 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d40bf-33f1-4ebb-b22f-651d69625df4 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:01.080327 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d40bf-33f1-4ebb-b22f-651d69625df4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:01.080662 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d40bf-33f1-4ebb-b22f-651d69625df4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:01.082021 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d40bf-33f1-4ebb-b22f-651d69625df4 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:01.086404 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 435/856] 10.222.0.27 () {56 vars in 1114 bytes} [Sun Sep 4 00:49:00 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:01.087176 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d40bf-33f1-4ebb-b22f-651d69625df4 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:01.101532 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:01.119258 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:01.141049 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 422/857] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:49:00 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 04 00:49:01.142911 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf482302-18a5-4674-bc32-3d87dfe31f5d None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:01.143198 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf482302-18a5-4674-bc32-3d87dfe31f5d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:01.143482 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf482302-18a5-4674-bc32-3d87dfe31f5d None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:01.144861 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 423/858] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:49:01.147576 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf482302-18a5-4674-bc32-3d87dfe31f5d None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:01.191142 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d40bf-33f1-4ebb-b22f-651d69625df4 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:01.226612 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ba62c-5352-49f0-89ee-93821aa9f703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:01.227212 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ba62c-5352-49f0-89ee-93821aa9f703 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:01.227493 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ba62c-5352-49f0-89ee-93821aa9f703 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:01.227769 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ba62c-5352-49f0-89ee-93821aa9f703 None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:01.228472 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ba62c-5352-49f0-89ee-93821aa9f703 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:01.228792 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ba62c-5352-49f0-89ee-93821aa9f703 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:01.230379 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ba62c-5352-49f0-89ee-93821aa9f703 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:01.242393 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 436/859] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:49:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:01.243195 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ba62c-5352-49f0-89ee-93821aa9f703 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:01.251679 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf482302-18a5-4674-bc32-3d87dfe31f5d None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:01.285029 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4dbe07-999a-40ac-94c5-b93904b00b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:01.285625 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4dbe07-999a-40ac-94c5-b93904b00b85 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:01.285905 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4dbe07-999a-40ac-94c5-b93904b00b85 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:01.286522 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4dbe07-999a-40ac-94c5-b93904b00b85 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:01.287138 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4dbe07-999a-40ac-94c5-b93904b00b85 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:01.287447 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4dbe07-999a-40ac-94c5-b93904b00b85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:01.288879 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4dbe07-999a-40ac-94c5-b93904b00b85 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:01.293467 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 424/860] 10.222.0.27 () {56 vars in 1114 bytes} [Sun Sep 4 00:49:01 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:01.294656 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0331cbc8-158d-4310-856e-37a2a34b804e None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:01.294939 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0331cbc8-158d-4310-856e-37a2a34b804e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:01.295210 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0331cbc8-158d-4310-856e-37a2a34b804e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:01.315515 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-0331cbc8-158d-4310-856e-37a2a34b804e None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:01.316830 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0331cbc8-158d-4310-856e-37a2a34b804e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:01.343069 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:01.354043 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:01.365709 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ba62c-5352-49f0-89ee-93821aa9f703 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:01.403595 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d8fd55-58c4-4f09-84e6-c866891c3971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:01.404241 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8fd55-58c4-4f09-84e6-c866891c3971 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:01.404554 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8fd55-58c4-4f09-84e6-c866891c3971 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:01.404749 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 425/861] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:49:01.404831 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8fd55-58c4-4f09-84e6-c866891c3971 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:01.405628 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0331cbc8-158d-4310-856e-37a2a34b804e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:01.417930 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d8fd55-58c4-4f09-84e6-c866891c3971 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:01.419464 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d8fd55-58c4-4f09-84e6-c866891c3971 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:01.436983 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-b9d8fd55-58c4-4f09-84e6-c866891c3971 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:49:01.436983 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:01.436983 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:01.436983 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:01.436983 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:01.436983 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:01.436983 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:01.436983 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:01.436983 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:01.436983 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:01.436983 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:01.436983 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:01.436983 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:49:01.436983 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:49:01.436983 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:49:01.436983 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:49:01.438096 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:01.438096 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:01.438096 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:01.438096 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:01.438096 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:01.438096 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:01.438096 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:01.438096 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:01.438096 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:01.438096 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:01.438096 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:01.438096 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:01.438096 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:01.438096 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:01.438096 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:01.438096 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:01.438096 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:49:01.439500 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:49:01.439500 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:49:01.439500 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:01.444113 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 437/862] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:49:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:01.444869 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d8fd55-58c4-4f09-84e6-c866891c3971 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:01.530379 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0331cbc8-158d-4310-856e-37a2a34b804e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:01.556838 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d8fd55-58c4-4f09-84e6-c866891c3971 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:01.567282 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19632a39-f599-420e-baee-29d39d2a9ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:01.567985 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19632a39-f599-420e-baee-29d39d2a9ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:01.568331 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19632a39-f599-420e-baee-29d39d2a9ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:01.568719 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19632a39-f599-420e-baee-29d39d2a9ea7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:01.569518 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19632a39-f599-420e-baee-29d39d2a9ea7 None admin] RBAC: Adding query filter params (role.id=6e3e398ba29e436e8532a45d2fdf1321, scope.project.id=5100a8d316d243fb8f82e9044e1b3b06, user.id=d6d01ca673a1404c9b4cea51036c595c) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:01.569886 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19632a39-f599-420e-baee-29d39d2a9ea7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:01.571364 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19632a39-f599-420e-baee-29d39d2a9ea7 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:01.577778 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 426/863] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:49:01 2022] GET /identity/v3/role_assignments?user.id=d6d01ca673a1404c9b4cea51036c595c&scope.project.id=5100a8d316d243fb8f82e9044e1b3b06&role.id=6e3e398ba29e436e8532a45d2fdf1321 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:01.593579 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc36dac8-14d1-4a2f-a416-18f079f85f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:01.594724 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36dac8-14d1-4a2f-a416-18f079f85f7f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:01.595623 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36dac8-14d1-4a2f-a416-18f079f85f7f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:01.595910 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36dac8-14d1-4a2f-a416-18f079f85f7f None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:01.596616 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc36dac8-14d1-4a2f-a416-18f079f85f7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:01.596926 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc36dac8-14d1-4a2f-a416-18f079f85f7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:01.598350 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc36dac8-14d1-4a2f-a416-18f079f85f7f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:01.613970 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 438/864] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:49:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:49:01.621705 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19632a39-f599-420e-baee-29d39d2a9ea7 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:01.648823 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:01.683028 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc36dac8-14d1-4a2f-a416-18f079f85f7f None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:01.686217 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:01.749578 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19632a39-f599-420e-baee-29d39d2a9ea7 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:01.795765 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6553bad6-8297-41d8-8a24-7b5df828dcd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:01.796402 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553bad6-8297-41d8-8a24-7b5df828dcd2 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:01.796754 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553bad6-8297-41d8-8a24-7b5df828dcd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:01.797033 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553bad6-8297-41d8-8a24-7b5df828dcd2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:01.804258 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6553bad6-8297-41d8-8a24-7b5df828dcd2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:01.805792 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6553bad6-8297-41d8-8a24-7b5df828dcd2 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:01.818006 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-6553bad6-8297-41d8-8a24-7b5df828dcd2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:49:01.818006 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:01.818006 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:01.818006 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:01.818006 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:01.818006 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:01.818006 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:01.818006 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:01.818006 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:01.818006 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:01.818006 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:01.818006 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:01.818006 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:49:01.818006 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:49:01.818006 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:49:01.818901 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:49:01.818901 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:01.818901 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:01.818901 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:01.818901 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:01.818901 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:01.818901 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:01.818901 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:01.818901 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:01.818901 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:01.818901 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:01.818901 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:01.818901 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:01.818901 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:01.818901 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:01.818901 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:01.818901 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:01.819761 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:49:01.819761 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:49:01.819761 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:49:01.819761 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:49:01.819761 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:49:01.819761 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:49:01.819761 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:49:01.819761 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:01.819761 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 427/865] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:49:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:01.821852 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc36dac8-14d1-4a2f-a416-18f079f85f7f None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:01.825481 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6553bad6-8297-41d8-8a24-7b5df828dcd2 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:01.867884 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c3772-8596-4fe1-aa0f-c9526ef882fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:01.868553 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c3772-8596-4fe1-aa0f-c9526ef882fe None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:01.868835 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c3772-8596-4fe1-aa0f-c9526ef882fe None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:01.869136 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c3772-8596-4fe1-aa0f-c9526ef882fe None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:01.870082 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c3772-8596-4fe1-aa0f-c9526ef882fe None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:01.871044 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c3772-8596-4fe1-aa0f-c9526ef882fe None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:01.883385 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-a10c3772-8596-4fe1-aa0f-c9526ef882fe None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 04 00:49:01.883385 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:01.883385 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 00:49:01.883385 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 00:49:01.883385 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 00:49:01.883385 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 00:49:01.883385 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 00:49:01.883385 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:49:01.883385 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 04 00:49:01.883385 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:01.883385 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 00:49:01.883385 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:01.883385 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:01.883385 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:01.883385 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:01.883385 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:01.888194 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:01.888194 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:01.888194 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:01.888194 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:01.888194 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:01.888194 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:01.888194 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:01.888194 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 00:49:01.888194 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 00:49:01.888194 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 00:49:01.888194 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 00:49:01.888194 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:01.888194 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:01.888194 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:01.888194 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:01.888194 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:01.888194 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:01.889114 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:01.889114 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:01.889114 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:01.889114 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:01.889114 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:01.889114 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:01.889114 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:01.889114 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:01.889114 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:01.889114 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:01.889114 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 00:49:01.889114 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:49:01.889114 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 04 00:49:01.889114 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:01.889114 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 439/866] 10.222.0.27 () {56 vars in 1132 bytes} [Sun Sep 4 00:49:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:49:01.889908 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c3772-8596-4fe1-aa0f-c9526ef882fe None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:01.930173 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:01.951961 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6553bad6-8297-41d8-8a24-7b5df828dcd2 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:01.977580 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:01.995792 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95709a53-af55-4edd-98e2-47acdfccc569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:01.996883 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95709a53-af55-4edd-98e2-47acdfccc569 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:01.997742 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95709a53-af55-4edd-98e2-47acdfccc569 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:01.998173 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95709a53-af55-4edd-98e2-47acdfccc569 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:01.999274 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95709a53-af55-4edd-98e2-47acdfccc569 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:01.999762 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95709a53-af55-4edd-98e2-47acdfccc569 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:02.002040 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95709a53-af55-4edd-98e2-47acdfccc569 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:02.017541 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c3772-8596-4fe1-aa0f-c9526ef882fe None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:02.025450 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 428/867] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:49:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:49:02.026735 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95709a53-af55-4edd-98e2-47acdfccc569 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:02.056865 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae98202-8641-4b34-99e9-7f667d1bbafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:02.057356 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae98202-8641-4b34-99e9-7f667d1bbafa None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:02.057643 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae98202-8641-4b34-99e9-7f667d1bbafa None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:02.057919 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae98202-8641-4b34-99e9-7f667d1bbafa None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:02.058550 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae98202-8641-4b34-99e9-7f667d1bbafa None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:02.058952 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae98202-8641-4b34-99e9-7f667d1bbafa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:02.060364 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae98202-8641-4b34-99e9-7f667d1bbafa None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:02.066147 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 440/868] 10.222.0.27 () {56 vars in 1126 bytes} [Sun Sep 4 00:49:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:02.071250 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae98202-8641-4b34-99e9-7f667d1bbafa None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:02.154969 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95709a53-af55-4edd-98e2-47acdfccc569 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:02.190680 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae98202-8641-4b34-99e9-7f667d1bbafa None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:02.198200 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12761b-826d-47db-b7dc-9b4eda8f62f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:02.199501 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12761b-826d-47db-b7dc-9b4eda8f62f6 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:02.199996 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12761b-826d-47db-b7dc-9b4eda8f62f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:02.200523 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12761b-826d-47db-b7dc-9b4eda8f62f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:02.201855 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be12761b-826d-47db-b7dc-9b4eda8f62f6 None admin] RBAC: Adding query filter params (role.id=c8a59d34ee2f4cefa6940cac778260c7, scope.project.id=5100a8d316d243fb8f82e9044e1b3b06, user.id=2297d8b3fe8e47eb9fd39a6e3eea8f54) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:02.202378 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be12761b-826d-47db-b7dc-9b4eda8f62f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:02.205327 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be12761b-826d-47db-b7dc-9b4eda8f62f6 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:02.217217 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 429/869] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:49:02 2022] GET /identity/v3/role_assignments?user.id=2297d8b3fe8e47eb9fd39a6e3eea8f54&scope.project.id=5100a8d316d243fb8f82e9044e1b3b06&role.id=c8a59d34ee2f4cefa6940cac778260c7 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:02.248552 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8a8947-b541-4e8a-b7ec-74d40b083894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:02.249568 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a8947-b541-4e8a-b7ec-74d40b083894 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:02.250059 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a8947-b541-4e8a-b7ec-74d40b083894 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:02.250730 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a8947-b541-4e8a-b7ec-74d40b083894 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:02.251939 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8a8947-b541-4e8a-b7ec-74d40b083894 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:02.252601 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8a8947-b541-4e8a-b7ec-74d40b083894 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:02.254291 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8a8947-b541-4e8a-b7ec-74d40b083894 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:02.260299 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 441/870] 10.222.0.27 () {56 vars in 1126 bytes} [Sun Sep 4 00:49:02 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:02.260655 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:03.326383 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9da83a-1cc1-42f6-95d5-c0bda53efe0a None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:03.326941 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9da83a-1cc1-42f6-95d5-c0bda53efe0a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:03.326941 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9da83a-1cc1-42f6-95d5-c0bda53efe0a None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:03.328590 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 430/871] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:49:03.335156 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691dfc7-9411-4bd3-92a9-47ba9a7c2a91 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:03.335616 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691dfc7-9411-4bd3-92a9-47ba9a7c2a91 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:03.335723 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691dfc7-9411-4bd3-92a9-47ba9a7c2a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:03.356737 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-6691dfc7-9411-4bd3-92a9-47ba9a7c2a91 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:03.357955 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-6691dfc7-9411-4bd3-92a9-47ba9a7c2a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:03.429045 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 442/872] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:03 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:49:03.706614 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9da83a-1cc1-42f6-95d5-c0bda53efe0a None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:03.711006 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:03.730769 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78df65c-252b-4059-9e29-fa52944d6427 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:03.731052 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78df65c-252b-4059-9e29-fa52944d6427 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:03.731330 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78df65c-252b-4059-9e29-fa52944d6427 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:03.733122 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 443/873] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 00:49:03.738733 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a33a0e6-871f-46b5-90c1-aeef0a284f1b None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:03.739014 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a33a0e6-871f-46b5-90c1-aeef0a284f1b None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:03.739291 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a33a0e6-871f-46b5-90c1-aeef0a284f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:03.759901 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-2a33a0e6-871f-46b5-90c1-aeef0a284f1b None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:03.761532 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2a33a0e6-871f-46b5-90c1-aeef0a284f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:03.833848 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 444/874] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:03 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 00:49:03.841770 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9da83a-1cc1-42f6-95d5-c0bda53efe0a None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:03.881341 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc162f-b809-4cff-8154-7b9580d51bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:03.882286 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc162f-b809-4cff-8154-7b9580d51bef None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:03.882739 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc162f-b809-4cff-8154-7b9580d51bef None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:03.890437 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc162f-b809-4cff-8154-7b9580d51bef None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:03.890437 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc162f-b809-4cff-8154-7b9580d51bef None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:03.890437 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc162f-b809-4cff-8154-7b9580d51bef None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:03.982465 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 431/875] 10.222.0.27 () {60 vars in 1141 bytes} [Sun Sep 4 00:49:03 2022] POST /identity/v3/services => generated 251 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:49:04.119285 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a33a0e6-871f-46b5-90c1-aeef0a284f1b None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:04.177160 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:04.208542 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d11969-6090-4b32-b168-acc8e2e7d04f None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:04.208954 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d11969-6090-4b32-b168-acc8e2e7d04f None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:04.209450 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d11969-6090-4b32-b168-acc8e2e7d04f None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:04.210909 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 432/876] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:49:04.216036 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f994b7c2-b44b-4e21-942f-dfc6068a1072 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:04.216322 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f994b7c2-b44b-4e21-942f-dfc6068a1072 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:04.216632 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f994b7c2-b44b-4e21-942f-dfc6068a1072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:04.237332 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a33a0e6-871f-46b5-90c1-aeef0a284f1b None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:04.237638 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f994b7c2-b44b-4e21-942f-dfc6068a1072 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:04.238830 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f994b7c2-b44b-4e21-942f-dfc6068a1072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:04.280892 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af90b35-a352-42cb-9a1b-e241daf575b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:04.281547 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af90b35-a352-42cb-9a1b-e241daf575b0 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:04.281823 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af90b35-a352-42cb-9a1b-e241daf575b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:04.282095 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af90b35-a352-42cb-9a1b-e241daf575b0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:04.293718 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af90b35-a352-42cb-9a1b-e241daf575b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:49:04.299528 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:04.300108 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-0af90b35-a352-42cb-9a1b-e241daf575b0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 04 00:49:04.300108 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:04.300108 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:04.300108 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:04.300108 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:04.300108 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:04.300108 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:04.300108 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:04.300108 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:04.300108 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:04.300108 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:04.300108 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:04.300108 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:49:04.300108 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:49:04.300108 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:49:04.300108 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:49:04.301062 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:49:04.301062 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:49:04.301062 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:49:04.301062 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:49:04.301062 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:04.301062 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:04.301062 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:04.301062 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:04.301062 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:04.301062 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:04.301062 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:04.301062 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:04.301062 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:04.301062 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:04.301062 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:04.301062 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:04.301904 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:04.301904 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:04.301904 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:04.301904 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:04.301904 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:49:04.301904 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:49:04.301904 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:49:04.301904 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:49:04.301904 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:49:04.301904 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:49:04.301904 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 04 00:49:04.301904 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:04.301904 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 445/877] 10.222.0.27 () {56 vars in 1120 bytes} [Sun Sep 4 00:49:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:04.303101 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a909be-221b-49ac-843a-933bdab9390c None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:04.303101 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a909be-221b-49ac-843a-933bdab9390c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:04.303398 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a909be-221b-49ac-843a-933bdab9390c None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:04.308715 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 446/878] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:49:04.308715 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a909be-221b-49ac-843a-933bdab9390c None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:04.316485 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 433/879] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:04 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 00:49:04.318605 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a85a33-b9e1-477d-a7e8-776ab8169f54 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:04.318605 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a85a33-b9e1-477d-a7e8-776ab8169f54 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:04.318605 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a85a33-b9e1-477d-a7e8-776ab8169f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:04.341518 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c4a85a33-b9e1-477d-a7e8-776ab8169f54 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:04.343613 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c4a85a33-b9e1-477d-a7e8-776ab8169f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:04.424218 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a909be-221b-49ac-843a-933bdab9390c None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:04.429187 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 434/880] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:04 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:49:04.440904 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:04.466421 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f0f696-a647-4524-843f-0861d97e29d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:04.467327 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0f696-a647-4524-843f-0861d97e29d7 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:04.467783 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0f696-a647-4524-843f-0861d97e29d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:04.468205 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0f696-a647-4524-843f-0861d97e29d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:04.469669 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f0f696-a647-4524-843f-0861d97e29d7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:04.470122 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f0f696-a647-4524-843f-0861d97e29d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:04.471654 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f0f696-a647-4524-843f-0861d97e29d7 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:04.480612 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 447/881] 10.222.0.27 () {56 vars in 1115 bytes} [Sun Sep 4 00:49:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:04.486179 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a85a33-b9e1-477d-a7e8-776ab8169f54 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:04.587548 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a85a33-b9e1-477d-a7e8-776ab8169f54 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:04.612376 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:04.623018 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b5b33-ce64-481d-9c01-7757a46f91b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:04.623657 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b5b33-ce64-481d-9c01-7757a46f91b4 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:04.623939 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b5b33-ce64-481d-9c01-7757a46f91b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:04.624218 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b5b33-ce64-481d-9c01-7757a46f91b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:04.629077 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f0f696-a647-4524-843f-0861d97e29d7 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:04.638503 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2b5b33-ce64-481d-9c01-7757a46f91b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:04.639958 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2b5b33-ce64-481d-9c01-7757a46f91b4 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:04.656067 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-7e2b5b33-ce64-481d-9c01-7757a46f91b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:49:04.656067 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:04.656067 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:04.656067 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:04.656067 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:04.656067 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:04.656067 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:04.656067 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:04.656067 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:04.656067 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:04.656067 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:04.656067 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:04.656067 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:49:04.656067 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:49:04.656067 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:49:04.656067 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:49:04.657916 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:04.657916 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:04.657916 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:04.657916 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:04.657916 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:04.657916 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:04.657916 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:04.657916 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:04.657916 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:04.657916 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:04.657916 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:04.657916 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:04.657916 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:04.657916 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:04.657916 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:04.657916 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:04.657916 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:49:04.658802 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:49:04.658802 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:49:04.658802 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:04.658802 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 435/882] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:49:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:04.665245 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b5b33-ce64-481d-9c01-7757a46f91b4 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:04.742762 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f0f696-a647-4524-843f-0861d97e29d7 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:04.775328 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b5b33-ce64-481d-9c01-7757a46f91b4 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:04.777682 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6b0234-4330-4f1c-8123-1f71e91b97d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:04.778308 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b0234-4330-4f1c-8123-1f71e91b97d0 None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:04.778587 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b0234-4330-4f1c-8123-1f71e91b97d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:04.778857 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b0234-4330-4f1c-8123-1f71e91b97d0 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:04.779508 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6b0234-4330-4f1c-8123-1f71e91b97d0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:04.780899 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6b0234-4330-4f1c-8123-1f71e91b97d0 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:04.811300 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29880291-876d-40c1-9b75-3c92e1e06fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:04.811903 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29880291-876d-40c1-9b75-3c92e1e06fab None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:04.812184 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29880291-876d-40c1-9b75-3c92e1e06fab None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:04.812449 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29880291-876d-40c1-9b75-3c92e1e06fab None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:04.813736 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29880291-876d-40c1-9b75-3c92e1e06fab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:04.814036 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29880291-876d-40c1-9b75-3c92e1e06fab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:04.815373 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29880291-876d-40c1-9b75-3c92e1e06fab None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:04.826942 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 436/883] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:49:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:04.827720 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29880291-876d-40c1-9b75-3c92e1e06fab None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:04.861547 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:04.873351 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 448/884] 10.222.0.27 () {60 vars in 1141 bytes} [Sun Sep 4 00:49:04 2022] POST /identity/v3/services => generated 269 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 00:49:04.874542 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6b0234-4330-4f1c-8123-1f71e91b97d0 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:04.918098 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:04.960432 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29880291-876d-40c1-9b75-3c92e1e06fab None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:05.016374 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6b0234-4330-4f1c-8123-1f71e91b97d0 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:05.017335 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7923a6d-5191-43b4-9280-6f13695a0578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:05.018383 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7923a6d-5191-43b4-9280-6f13695a0578 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:05.019738 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7923a6d-5191-43b4-9280-6f13695a0578 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:05.020207 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7923a6d-5191-43b4-9280-6f13695a0578 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:05.020824 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7923a6d-5191-43b4-9280-6f13695a0578 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:05.022190 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7923a6d-5191-43b4-9280-6f13695a0578 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:05.031968 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-b7923a6d-5191-43b4-9280-6f13695a0578 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 04 00:49:05.031968 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:05.031968 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 00:49:05.031968 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 00:49:05.031968 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 00:49:05.031968 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 00:49:05.031968 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 00:49:05.031968 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:49:05.031968 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 04 00:49:05.031968 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:05.031968 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 00:49:05.031968 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:05.031968 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:05.031968 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:05.031968 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:05.031968 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:05.031968 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:05.033276 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:05.033276 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:05.033276 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:05.033276 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:05.033276 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:05.033276 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:05.033276 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 00:49:05.033276 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 00:49:05.033276 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 00:49:05.033276 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 00:49:05.033276 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:05.033276 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:05.033276 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:05.033276 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:05.033276 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:05.033276 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:05.034121 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:05.034121 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:05.034121 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:05.034121 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:05.034121 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:05.034121 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:05.034121 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:05.034121 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:05.034121 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:05.034121 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:05.034121 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 00:49:05.034121 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:49:05.034121 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 04 00:49:05.034121 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:05.034121 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 437/885] 10.222.0.27 () {56 vars in 1126 bytes} [Sun Sep 4 00:49:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:05.039605 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7923a6d-5191-43b4-9280-6f13695a0578 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:05.070526 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e79627-176b-4a37-bdf7-0db078432db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:05.071175 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e79627-176b-4a37-bdf7-0db078432db0 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:05.071459 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e79627-176b-4a37-bdf7-0db078432db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:05.071736 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e79627-176b-4a37-bdf7-0db078432db0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:05.081854 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e79627-176b-4a37-bdf7-0db078432db0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:05.084361 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e79627-176b-4a37-bdf7-0db078432db0 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:05.096709 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-63e79627-176b-4a37-bdf7-0db078432db0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 04 00:49:05.096709 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:05.096709 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:05.096709 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:05.096709 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:05.096709 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:05.096709 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:05.096709 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:05.096709 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:05.096709 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:05.096709 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:05.096709 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:05.096709 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:49:05.096709 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:49:05.096709 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:49:05.096709 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:49:05.097658 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:05.097658 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:05.097658 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:49:05.097658 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:49:05.097658 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:49:05.097658 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:49:05.097658 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:05.097658 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:05.097658 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:05.097658 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:05.097658 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:05.097658 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:05.097658 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:05.097658 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:05.097658 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:05.097658 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:05.097658 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:05.097658 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:05.098580 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:05.098580 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:05.098580 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:49:05.098580 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:49:05.098580 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:49:05.098580 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:49:05.098580 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:49:05.098580 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:49:05.098580 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 04 00:49:05.098580 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:05.098580 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 449/886] 10.222.0.27 () {56 vars in 1117 bytes} [Sun Sep 4 00:49:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:05.105414 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e79627-176b-4a37-bdf7-0db078432db0 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:05.156959 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:05.163099 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7923a6d-5191-43b4-9280-6f13695a0578 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:05.201147 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e91f4-d239-4ce5-99a4-cd71d0c48750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:05.201814 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e91f4-d239-4ce5-99a4-cd71d0c48750 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:05.202099 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e91f4-d239-4ce5-99a4-cd71d0c48750 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:05.202381 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e91f4-d239-4ce5-99a4-cd71d0c48750 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:05.203007 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035e91f4-d239-4ce5-99a4-cd71d0c48750 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:05.203338 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035e91f4-d239-4ce5-99a4-cd71d0c48750 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:05.205253 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035e91f4-d239-4ce5-99a4-cd71d0c48750 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:05.214726 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 438/887] 10.222.0.27 () {56 vars in 1122 bytes} [Sun Sep 4 00:49:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:49:05.216392 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e91f4-d239-4ce5-99a4-cd71d0c48750 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:05.252142 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:05.264492 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e79627-176b-4a37-bdf7-0db078432db0 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:05.315767 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d1625-f890-44f5-9ce7-6260990181f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:05.317174 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d1625-f890-44f5-9ce7-6260990181f9 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:05.317764 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d1625-f890-44f5-9ce7-6260990181f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:05.318316 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d1625-f890-44f5-9ce7-6260990181f9 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:05.319455 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93d1625-f890-44f5-9ce7-6260990181f9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:05.319913 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93d1625-f890-44f5-9ce7-6260990181f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:05.322589 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93d1625-f890-44f5-9ce7-6260990181f9 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:05.330488 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e91f4-d239-4ce5-99a4-cd71d0c48750 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:05.348566 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 450/888] 10.222.0.27 () {56 vars in 1149 bytes} [Sun Sep 4 00:49:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:05.355023 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d1625-f890-44f5-9ce7-6260990181f9 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:05.381528 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182795ac-ac60-4d5a-a408-4e050e4587ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:05.382209 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182795ac-ac60-4d5a-a408-4e050e4587ea None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:05.382402 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182795ac-ac60-4d5a-a408-4e050e4587ea None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:05.382674 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182795ac-ac60-4d5a-a408-4e050e4587ea None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:05.383300 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182795ac-ac60-4d5a-a408-4e050e4587ea None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:05.383603 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182795ac-ac60-4d5a-a408-4e050e4587ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:05.385025 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182795ac-ac60-4d5a-a408-4e050e4587ea None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:05.396646 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 439/889] 10.222.0.27 () {56 vars in 1122 bytes} [Sun Sep 4 00:49:05 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:49:05.481461 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d1625-f890-44f5-9ce7-6260990181f9 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:05.521341 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d7961-ad85-4741-a4c4-283f82edcf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:05.521956 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d7961-ad85-4741-a4c4-283f82edcf3f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:05.522344 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d7961-ad85-4741-a4c4-283f82edcf3f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:05.522622 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d7961-ad85-4741-a4c4-283f82edcf3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:05.548903 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:05.601508 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d7961-ad85-4741-a4c4-283f82edcf3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:05.603514 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d7961-ad85-4741-a4c4-283f82edcf3f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:05.712184 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 451/890] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:49:05 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 04 00:49:05.717836 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182795ac-ac60-4d5a-a408-4e050e4587ea None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:05.745134 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:05.827582 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182795ac-ac60-4d5a-a408-4e050e4587ea None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:05.863483 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e48b465-9374-4ef0-924f-f954136a7059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:05.864484 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e48b465-9374-4ef0-924f-f954136a7059 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:05.864970 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e48b465-9374-4ef0-924f-f954136a7059 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:05.865461 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e48b465-9374-4ef0-924f-f954136a7059 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:05.876654 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e48b465-9374-4ef0-924f-f954136a7059 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:05.878385 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e48b465-9374-4ef0-924f-f954136a7059 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:05.894277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-3e48b465-9374-4ef0-924f-f954136a7059 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:49:05.894277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:05.894277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:05.894277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:05.894277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:05.894277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:05.894277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:05.894277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:05.894277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:05.894277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:05.894277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:05.894277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:05.894277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:49:05.894277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:49:05.894277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:49:05.894277 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:49:05.905079 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:05.905079 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:05.905079 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:05.905079 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:05.905079 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:05.905079 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:05.905079 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:05.905079 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:05.905079 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:05.905079 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:05.905079 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:05.905079 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:05.905079 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:05.905079 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:05.905079 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:05.905079 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:05.905079 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:49:05.906112 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:49:05.906112 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:49:05.906112 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:05.906112 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 440/891] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:49:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:05.906354 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d7961-ad85-4741-a4c4-283f82edcf3f None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:05.960498 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:05.992772 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5142d-3fda-402b-983a-4a03611b84d9 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:05.993405 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5142d-3fda-402b-983a-4a03611b84d9 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:05.993902 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5142d-3fda-402b-983a-4a03611b84d9 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:05.995759 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 441/892] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 00:49:06.002400 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a2b11-90a8-4cf9-b572-1c44c6b87d7c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:06.002935 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a2b11-90a8-4cf9-b572-1c44c6b87d7c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:06.003403 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a2b11-90a8-4cf9-b572-1c44c6b87d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:06.014625 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d7961-ad85-4741-a4c4-283f82edcf3f None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:06.021930 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-2e6a2b11-90a8-4cf9-b572-1c44c6b87d7c None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:06.023485 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2e6a2b11-90a8-4cf9-b572-1c44c6b87d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:06.054396 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd26be2-edc3-4358-a0b5-b817c1c01429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:06.055581 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd26be2-edc3-4358-a0b5-b817c1c01429 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:06.056328 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd26be2-edc3-4358-a0b5-b817c1c01429 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:06.057229 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd26be2-edc3-4358-a0b5-b817c1c01429 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:06.058344 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd26be2-edc3-4358-a0b5-b817c1c01429 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:06.058890 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd26be2-edc3-4358-a0b5-b817c1c01429 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:06.060725 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd26be2-edc3-4358-a0b5-b817c1c01429 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:06.083025 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 452/893] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:49:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:06.085289 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd26be2-edc3-4358-a0b5-b817c1c01429 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:06.116484 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 442/894] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:06 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:49:06.199246 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd26be2-edc3-4358-a0b5-b817c1c01429 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:06.210682 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:06.235279 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7211b300-1b54-443b-8f71-93235e4aa3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:06.245147 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211b300-1b54-443b-8f71-93235e4aa3c3 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:06.245147 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211b300-1b54-443b-8f71-93235e4aa3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:06.245147 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211b300-1b54-443b-8f71-93235e4aa3c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:06.245147 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7211b300-1b54-443b-8f71-93235e4aa3c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:06.245147 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7211b300-1b54-443b-8f71-93235e4aa3c3 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:06.257710 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-7211b300-1b54-443b-8f71-93235e4aa3c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:49:06.257710 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:06.257710 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:06.257710 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:06.257710 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:06.257710 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:06.257710 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:06.257710 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:06.257710 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:06.257710 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:06.257710 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:06.257710 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:06.257710 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:49:06.257710 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:49:06.257710 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:49:06.258606 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:49:06.258606 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:06.258606 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:06.258606 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:06.258606 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:06.258606 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:06.258606 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:06.258606 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:06.258606 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:06.258606 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:06.258606 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:06.258606 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:06.258606 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:06.258606 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:06.258606 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:06.258606 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:06.258606 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:06.259772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:49:06.259772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:49:06.259772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:49:06.259772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:49:06.259772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:49:06.259772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:49:06.259772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:49:06.259772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:06.259772 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 453/895] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:49:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:49:06.267475 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a2b11-90a8-4cf9-b572-1c44c6b87d7c None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:06.276663 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:06.374051 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a2b11-90a8-4cf9-b572-1c44c6b87d7c None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:06.402722 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7211b300-1b54-443b-8f71-93235e4aa3c3 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:06.410975 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ef633-27a3-47b9-88b6-5dfdd21c9f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:06.412204 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ef633-27a3-47b9-88b6-5dfdd21c9f0b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:06.412710 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ef633-27a3-47b9-88b6-5dfdd21c9f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:06.413182 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ef633-27a3-47b9-88b6-5dfdd21c9f0b None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:06.414425 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434ef633-27a3-47b9-88b6-5dfdd21c9f0b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:06.414945 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434ef633-27a3-47b9-88b6-5dfdd21c9f0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:06.416588 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434ef633-27a3-47b9-88b6-5dfdd21c9f0b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:06.434955 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 443/896] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:49:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:06.436525 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ef633-27a3-47b9-88b6-5dfdd21c9f0b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:06.506435 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:06.516385 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7211b300-1b54-443b-8f71-93235e4aa3c3 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:06.547337 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ef633-27a3-47b9-88b6-5dfdd21c9f0b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:06.549216 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d1ccf5-5157-4a43-8eff-7115fc1db6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:06.549936 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1ccf5-5157-4a43-8eff-7115fc1db6c7 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:06.550295 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1ccf5-5157-4a43-8eff-7115fc1db6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:06.550696 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1ccf5-5157-4a43-8eff-7115fc1db6c7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:06.551829 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d1ccf5-5157-4a43-8eff-7115fc1db6c7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:06.552820 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d1ccf5-5157-4a43-8eff-7115fc1db6c7 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:06.562470 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-01d1ccf5-5157-4a43-8eff-7115fc1db6c7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 04 00:49:06.562470 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:06.562470 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 00:49:06.562470 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 00:49:06.562470 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 00:49:06.562470 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 00:49:06.562470 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 00:49:06.562470 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:49:06.562470 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 04 00:49:06.562470 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:06.562470 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 00:49:06.562470 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:06.562470 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:06.562470 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:06.562470 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:06.562470 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:06.562470 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:06.563586 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:06.563586 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:06.563586 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:06.563586 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:06.563586 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:06.563586 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:06.563586 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 00:49:06.563586 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 00:49:06.563586 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 00:49:06.563586 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 00:49:06.563586 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:06.563586 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:06.563586 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:06.563586 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:06.563586 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:06.563586 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:06.564413 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:06.564413 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:06.564413 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:06.564413 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:06.564413 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:06.564413 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:06.564413 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:06.564413 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:06.564413 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:06.564413 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:06.564413 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 00:49:06.564413 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:49:06.564413 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 04 00:49:06.564413 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:06.568946 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 454/897] 10.222.0.27 () {56 vars in 1114 bytes} [Sun Sep 4 00:49:06 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:06.570064 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d1ccf5-5157-4a43-8eff-7115fc1db6c7 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:06.591753 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab35e19-4b85-47a8-96a6-628d5a506a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:06.592718 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab35e19-4b85-47a8-96a6-628d5a506a2a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:06.593178 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab35e19-4b85-47a8-96a6-628d5a506a2a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:06.593627 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab35e19-4b85-47a8-96a6-628d5a506a2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:06.595079 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab35e19-4b85-47a8-96a6-628d5a506a2a None admin] RBAC: Adding query filter params (role.id=91d340dbd1f142c5aa33ad640eca891c, scope.project.id=5100a8d316d243fb8f82e9044e1b3b06, user.id=d6d01ca673a1404c9b4cea51036c595c) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:06.595630 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab35e19-4b85-47a8-96a6-628d5a506a2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:06.597313 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab35e19-4b85-47a8-96a6-628d5a506a2a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:06.608587 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 444/898] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:49:06 2022] GET /identity/v3/role_assignments?user.id=d6d01ca673a1404c9b4cea51036c595c&scope.project.id=5100a8d316d243fb8f82e9044e1b3b06&role.id=91d340dbd1f142c5aa33ad640eca891c => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:49:06.608838 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:06.691463 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d1ccf5-5157-4a43-8eff-7115fc1db6c7 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:06.729885 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db634c1-6ed7-4a86-9586-bb7e437b3f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:06.730790 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db634c1-6ed7-4a86-9586-bb7e437b3f61 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:06.731216 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db634c1-6ed7-4a86-9586-bb7e437b3f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:06.731632 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db634c1-6ed7-4a86-9586-bb7e437b3f61 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:06.732579 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db634c1-6ed7-4a86-9586-bb7e437b3f61 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:06.733066 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db634c1-6ed7-4a86-9586-bb7e437b3f61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:06.735189 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db634c1-6ed7-4a86-9586-bb7e437b3f61 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:06.741061 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 455/899] 10.222.0.27 () {56 vars in 1114 bytes} [Sun Sep 4 00:49:06 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:06.747882 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab35e19-4b85-47a8-96a6-628d5a506a2a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:06.855562 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab35e19-4b85-47a8-96a6-628d5a506a2a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:06.885671 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:06.892693 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdae3da-6e91-47fd-952d-bf53413fa679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:06.893362 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdae3da-6e91-47fd-952d-bf53413fa679 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:06.893688 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdae3da-6e91-47fd-952d-bf53413fa679 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:06.893960 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdae3da-6e91-47fd-952d-bf53413fa679 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:06.894589 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdae3da-6e91-47fd-952d-bf53413fa679 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:06.894881 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdae3da-6e91-47fd-952d-bf53413fa679 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:06.896237 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdae3da-6e91-47fd-952d-bf53413fa679 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:06.901518 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 445/900] 10.222.0.27 () {56 vars in 1114 bytes} [Sun Sep 4 00:49:06 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:06.907545 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db634c1-6ed7-4a86-9586-bb7e437b3f61 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:06.943402 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c8ad8-d5d8-4375-8f4c-97a5ff6edfb8 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:06.943728 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c8ad8-d5d8-4375-8f4c-97a5ff6edfb8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:06.943966 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c8ad8-d5d8-4375-8f4c-97a5ff6edfb8 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:06.945610 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 446/901] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 00:49:06.950522 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6571cbf-825d-46f3-b4c4-60aad52e9478 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:06.950824 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6571cbf-825d-46f3-b4c4-60aad52e9478 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:06.951115 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6571cbf-825d-46f3-b4c4-60aad52e9478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:06.970621 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-b6571cbf-825d-46f3-b4c4-60aad52e9478 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:06.971827 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b6571cbf-825d-46f3-b4c4-60aad52e9478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:07.017944 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:07.023202 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db634c1-6ed7-4a86-9586-bb7e437b3f61 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:07.060608 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 447/902] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:06 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:49:07.072082 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a3e3a-c988-4812-9595-9daf38bdc00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:07.073086 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a3e3a-c988-4812-9595-9daf38bdc00e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:07.073565 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a3e3a-c988-4812-9595-9daf38bdc00e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:07.074034 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a3e3a-c988-4812-9595-9daf38bdc00e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:07.074995 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a3e3a-c988-4812-9595-9daf38bdc00e None admin] RBAC: Adding query filter params (interface=public, service_id=e535e2e89ef44fca9d992e2f4148fbc3, region_id=RegionOne) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:07.075530 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a3e3a-c988-4812-9595-9daf38bdc00e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:07.077149 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a3e3a-c988-4812-9595-9daf38bdc00e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:07.086851 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 456/903] 10.222.0.27 () {56 vars in 1251 bytes} [Sun Sep 4 00:49:06 2022] GET /identity/v3/endpoints?service_id=e535e2e89ef44fca9d992e2f4148fbc3&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:07.092200 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6571cbf-825d-46f3-b4c4-60aad52e9478 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:07.194619 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6571cbf-825d-46f3-b4c4-60aad52e9478 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:07.226196 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:07.230504 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa787f06-f0bb-4bd4-a3ec-6677889ffd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:07.231113 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa787f06-f0bb-4bd4-a3ec-6677889ffd31 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:07.231441 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa787f06-f0bb-4bd4-a3ec-6677889ffd31 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:07.231715 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa787f06-f0bb-4bd4-a3ec-6677889ffd31 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:07.232306 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa787f06-f0bb-4bd4-a3ec-6677889ffd31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:07.233739 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa787f06-f0bb-4bd4-a3ec-6677889ffd31 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:07.240533 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 448/904] 10.222.0.27 () {56 vars in 1087 bytes} [Sun Sep 4 00:49:07 2022] GET /identity/v3/services => generated 1268 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 00:49:07.345477 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a3e3a-c988-4812-9595-9daf38bdc00e None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:07.446667 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8fcd80-b0ac-45ed-b37c-e3ca2f77a755 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:07.447183 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8fcd80-b0ac-45ed-b37c-e3ca2f77a755 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:07.447183 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8fcd80-b0ac-45ed-b37c-e3ca2f77a755 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:07.448738 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 449/905] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:49:07.454580 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47884c2-b7c9-4d2e-84f7-549ca4f602fe None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:07.454875 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47884c2-b7c9-4d2e-84f7-549ca4f602fe None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:07.455439 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47884c2-b7c9-4d2e-84f7-549ca4f602fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:07.457142 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a3e3a-c988-4812-9595-9daf38bdc00e None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:07.478659 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-a47884c2-b7c9-4d2e-84f7-549ca4f602fe None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:07.480161 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a47884c2-b7c9-4d2e-84f7-549ca4f602fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:07.495536 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed2af4-7325-49ba-99f7-8107a02563da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:07.496760 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed2af4-7325-49ba-99f7-8107a02563da None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:07.497319 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed2af4-7325-49ba-99f7-8107a02563da None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:07.498235 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed2af4-7325-49ba-99f7-8107a02563da None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:07.499288 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed2af4-7325-49ba-99f7-8107a02563da None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:07.501774 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed2af4-7325-49ba-99f7-8107a02563da None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:07.514642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-f7ed2af4-7325-49ba-99f7-8107a02563da None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 04 00:49:07.514642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:07.514642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 00:49:07.514642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 00:49:07.514642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 00:49:07.514642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 00:49:07.514642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 00:49:07.514642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:49:07.514642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 04 00:49:07.514642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:07.514642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 00:49:07.514642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:07.514642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:07.514642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:07.514642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:07.514642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:07.515607 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:07.515607 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:07.515607 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:07.515607 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:07.515607 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:07.515607 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:07.515607 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:07.515607 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 00:49:07.515607 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 00:49:07.515607 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 00:49:07.515607 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 00:49:07.515607 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:07.515607 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:07.515607 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:07.515607 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:07.515607 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:07.515607 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:07.517485 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:07.517485 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:07.517485 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:07.517485 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:07.517485 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:07.517485 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:07.517485 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:07.517485 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:07.517485 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:07.517485 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:07.517485 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 00:49:07.517485 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:49:07.517485 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 04 00:49:07.517485 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:07.519003 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 457/906] 10.222.0.27 () {56 vars in 1132 bytes} [Sun Sep 4 00:49:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:07.523799 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed2af4-7325-49ba-99f7-8107a02563da None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:07.555040 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:07.576371 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 450/907] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:07 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:49:07.634520 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed2af4-7325-49ba-99f7-8107a02563da None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:07.670467 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550258a-ef7f-488d-aa12-4fb8a09dd356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:07.671079 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550258a-ef7f-488d-aa12-4fb8a09dd356 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:07.671360 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550258a-ef7f-488d-aa12-4fb8a09dd356 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:07.671636 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550258a-ef7f-488d-aa12-4fb8a09dd356 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:07.672311 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550258a-ef7f-488d-aa12-4fb8a09dd356 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:07.672714 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550258a-ef7f-488d-aa12-4fb8a09dd356 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:07.674164 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550258a-ef7f-488d-aa12-4fb8a09dd356 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:07.683172 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 458/908] 10.222.0.27 () {56 vars in 1126 bytes} [Sun Sep 4 00:49:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:49:07.685306 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47884c2-b7c9-4d2e-84f7-549ca4f602fe None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:07.795558 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47884c2-b7c9-4d2e-84f7-549ca4f602fe None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:07.813784 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:07.831659 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518b36a-5a20-47e9-baee-c306fd962852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:07.832269 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518b36a-5a20-47e9-baee-c306fd962852 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:07.832578 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518b36a-5a20-47e9-baee-c306fd962852 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:07.832863 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518b36a-5a20-47e9-baee-c306fd962852 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:07.833538 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d518b36a-5a20-47e9-baee-c306fd962852 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:07.833846 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d518b36a-5a20-47e9-baee-c306fd962852 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:07.835197 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d518b36a-5a20-47e9-baee-c306fd962852 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:07.846114 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 451/909] 10.222.0.27 () {56 vars in 1126 bytes} [Sun Sep 4 00:49:07 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:07.847591 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550258a-ef7f-488d-aa12-4fb8a09dd356 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:07.851529 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518b36a-5a20-47e9-baee-c306fd962852 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:07.946680 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:07.956988 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550258a-ef7f-488d-aa12-4fb8a09dd356 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:07.993479 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518b36a-5a20-47e9-baee-c306fd962852 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:07.997161 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455bb26-5681-46ee-9c70-c39bb909e0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:07.997773 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455bb26-5681-46ee-9c70-c39bb909e0a7 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:07.998058 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455bb26-5681-46ee-9c70-c39bb909e0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:07.998332 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455bb26-5681-46ee-9c70-c39bb909e0a7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:07.999030 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3455bb26-5681-46ee-9c70-c39bb909e0a7 None admin] RBAC: Adding query filter params (interface=public, service_id=cd084fb2f4c04224a8a838655996a1f8, region_id=RegionOne) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:07.999339 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3455bb26-5681-46ee-9c70-c39bb909e0a7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:08.001310 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3455bb26-5681-46ee-9c70-c39bb909e0a7 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:08.011212 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 459/910] 10.222.0.27 () {56 vars in 1251 bytes} [Sun Sep 4 00:49:07 2022] GET /identity/v3/endpoints?service_id=cd084fb2f4c04224a8a838655996a1f8&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:08.012102 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455bb26-5681-46ee-9c70-c39bb909e0a7 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:08.041967 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a71c691-570a-455e-8eb0-c14640a69c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:08.042936 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71c691-570a-455e-8eb0-c14640a69c95 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:08.043295 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71c691-570a-455e-8eb0-c14640a69c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:08.043902 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71c691-570a-455e-8eb0-c14640a69c95 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:08.045125 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a71c691-570a-455e-8eb0-c14640a69c95 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:08.047428 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a71c691-570a-455e-8eb0-c14640a69c95 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:08.118163 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455bb26-5681-46ee-9c70-c39bb909e0a7 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:08.120548 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 452/911] 10.222.0.27 () {60 vars in 1141 bytes} [Sun Sep 4 00:49:07 2022] POST /identity/v3/services => generated 241 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 00:49:08.173691 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748dd775-086a-4020-ac5e-0957bfad413a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:08.174319 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748dd775-086a-4020-ac5e-0957bfad413a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:08.174609 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748dd775-086a-4020-ac5e-0957bfad413a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:08.174883 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748dd775-086a-4020-ac5e-0957bfad413a None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:08.175568 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748dd775-086a-4020-ac5e-0957bfad413a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:08.177114 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748dd775-086a-4020-ac5e-0957bfad413a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:08.184656 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 460/912] 10.222.0.27 () {56 vars in 1087 bytes} [Sun Sep 4 00:49:08 2022] GET /identity/v3/services => generated 1497 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 00:49:08.612820 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172e89e-89fb-4876-85b7-97c36919f2f6 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:08.613513 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172e89e-89fb-4876-85b7-97c36919f2f6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:08.613513 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172e89e-89fb-4876-85b7-97c36919f2f6 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:08.614968 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 453/913] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:49:08.620159 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5f84c-faa4-438e-a8f1-fb44490cd465 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:08.620688 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5f84c-faa4-438e-a8f1-fb44490cd465 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:08.620840 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5f84c-faa4-438e-a8f1-fb44490cd465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:08.639461 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-feb5f84c-faa4-438e-a8f1-fb44490cd465 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:08.640751 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-feb5f84c-faa4-438e-a8f1-fb44490cd465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:08.655270 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:08.716969 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 461/914] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:49:08.986324 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172e89e-89fb-4876-85b7-97c36919f2f6 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:09.034574 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:09.097740 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172e89e-89fb-4876-85b7-97c36919f2f6 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:09.134222 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d4175a-9ff0-4f3d-9726-846a0fb4ffb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:09.134855 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4175a-9ff0-4f3d-9726-846a0fb4ffb0 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:09.135135 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4175a-9ff0-4f3d-9726-846a0fb4ffb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:09.135410 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4175a-9ff0-4f3d-9726-846a0fb4ffb0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:09.146197 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d4175a-9ff0-4f3d-9726-846a0fb4ffb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:49:09.152132 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-48d4175a-9ff0-4f3d-9726-846a0fb4ffb0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 04 00:49:09.152132 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:09.152132 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:09.152132 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:09.152132 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:09.152132 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:09.152132 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:09.152132 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:09.152132 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:09.152132 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:09.152132 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:09.152132 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:09.152132 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:49:09.152132 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:49:09.152132 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:49:09.152132 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:49:09.154230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:49:09.154230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:49:09.154230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:49:09.154230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:49:09.154230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:09.154230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:09.154230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:09.154230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:09.154230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:09.154230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:09.154230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:09.154230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:09.154230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:09.154230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:09.154230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:09.154230 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:09.156146 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:09.156146 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:09.156146 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:09.156146 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:09.156146 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:49:09.156146 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:49:09.156146 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:49:09.156146 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:49:09.156146 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:49:09.156146 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:49:09.156146 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 04 00:49:09.156146 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:09.156146 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 454/915] 10.222.0.27 () {56 vars in 1120 bytes} [Sun Sep 4 00:49:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:49:09.159612 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5f84c-faa4-438e-a8f1-fb44490cd465 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:09.274381 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5f84c-faa4-438e-a8f1-fb44490cd465 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:09.290092 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a305c86-5a0f-43d7-8666-0141454f3068 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:09.290381 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a305c86-5a0f-43d7-8666-0141454f3068 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:09.290681 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a305c86-5a0f-43d7-8666-0141454f3068 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:09.292283 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 455/916] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:49:09.297716 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9f282-9101-4914-94ee-16b697c30e56 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:09.298132 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9f282-9101-4914-94ee-16b697c30e56 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:09.298416 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9f282-9101-4914-94ee-16b697c30e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:09.314911 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5ab73-fd96-4c2c-a11e-4bb23565aa24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:09.315533 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5ab73-fd96-4c2c-a11e-4bb23565aa24 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:09.315817 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5ab73-fd96-4c2c-a11e-4bb23565aa24 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:09.316093 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5ab73-fd96-4c2c-a11e-4bb23565aa24 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:09.316798 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c5ab73-fd96-4c2c-a11e-4bb23565aa24 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:09.317108 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c5ab73-fd96-4c2c-a11e-4bb23565aa24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:09.318243 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1cf9f282-9101-4914-94ee-16b697c30e56 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:09.318492 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c5ab73-fd96-4c2c-a11e-4bb23565aa24 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:09.319853 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1cf9f282-9101-4914-94ee-16b697c30e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:09.330366 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 462/917] 10.222.0.27 () {56 vars in 1115 bytes} [Sun Sep 4 00:49:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:49:09.336232 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5ab73-fd96-4c2c-a11e-4bb23565aa24 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:09.346672 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:09.410521 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 456/918] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 00:49:09.454843 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5ab73-fd96-4c2c-a11e-4bb23565aa24 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:09.490309 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a91087-9d3b-4b15-87f1-c210f59babe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:09.490984 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a91087-9d3b-4b15-87f1-c210f59babe4 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:09.491272 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a91087-9d3b-4b15-87f1-c210f59babe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:09.491546 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a91087-9d3b-4b15-87f1-c210f59babe4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:09.503382 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a91087-9d3b-4b15-87f1-c210f59babe4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:09.504870 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a91087-9d3b-4b15-87f1-c210f59babe4 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:09.519737 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-f6a91087-9d3b-4b15-87f1-c210f59babe4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:49:09.519737 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:09.519737 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:09.519737 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:09.519737 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:09.519737 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:09.519737 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:09.519737 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:09.519737 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:09.519737 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:09.519737 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:09.519737 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:09.519737 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:49:09.519737 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:49:09.519737 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:49:09.519737 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:49:09.520709 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:09.520709 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:09.520709 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:09.520709 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:09.520709 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:09.520709 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:09.520709 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:09.520709 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:09.520709 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:09.520709 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:09.520709 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:09.520709 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:09.520709 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:09.520709 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:09.520709 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:09.520709 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:09.520709 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:49:09.522008 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:49:09.522008 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:49:09.522008 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:09.522008 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 463/919] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:49:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:49:09.526967 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf9f282-9101-4914-94ee-16b697c30e56 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:09.560779 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:09.634757 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf9f282-9101-4914-94ee-16b697c30e56 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:09.670705 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc36785-97bc-4694-a26f-9fa946c742a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:09.671663 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc36785-97bc-4694-a26f-9fa946c742a9 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:09.672944 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc36785-97bc-4694-a26f-9fa946c742a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:09.673425 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc36785-97bc-4694-a26f-9fa946c742a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:09.674266 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc36785-97bc-4694-a26f-9fa946c742a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:09.674768 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc36785-97bc-4694-a26f-9fa946c742a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:09.676347 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc36785-97bc-4694-a26f-9fa946c742a9 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:09.688905 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 457/920] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:49:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:09.694392 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a91087-9d3b-4b15-87f1-c210f59babe4 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:09.715571 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc36785-97bc-4694-a26f-9fa946c742a9 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:09.765121 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:09.811495 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:09.815180 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a91087-9d3b-4b15-87f1-c210f59babe4 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:09.820129 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc36785-97bc-4694-a26f-9fa946c742a9 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:09.855662 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d6397d-fd5a-4d4e-b34f-0f0539b73f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:09.856723 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6397d-fd5a-4d4e-b34f-0f0539b73f63 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:09.857014 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0eea8-84b8-4949-9f7b-dc3d6818750b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:09.857465 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6397d-fd5a-4d4e-b34f-0f0539b73f63 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:09.857666 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0eea8-84b8-4949-9f7b-dc3d6818750b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:09.857920 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0eea8-84b8-4949-9f7b-dc3d6818750b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:09.858195 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0eea8-84b8-4949-9f7b-dc3d6818750b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:09.858547 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6397d-fd5a-4d4e-b34f-0f0539b73f63 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:09.858774 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0eea8-84b8-4949-9f7b-dc3d6818750b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:09.860484 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0eea8-84b8-4949-9f7b-dc3d6818750b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:09.867554 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d6397d-fd5a-4d4e-b34f-0f0539b73f63 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:09.869360 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d6397d-fd5a-4d4e-b34f-0f0539b73f63 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:09.869477 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-4cb0eea8-84b8-4949-9f7b-dc3d6818750b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 04 00:49:09.869477 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:09.869477 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 00:49:09.869477 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 00:49:09.869477 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 00:49:09.869477 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 00:49:09.869477 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 00:49:09.869477 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:49:09.869477 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 04 00:49:09.869477 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:09.869477 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 00:49:09.869477 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:09.869477 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:09.869477 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:09.869477 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:09.869477 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:09.869477 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:09.870436 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:09.870436 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:09.870436 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:09.870436 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:09.870436 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:09.870436 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:09.870436 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 00:49:09.870436 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 00:49:09.870436 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 00:49:09.870436 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 00:49:09.870436 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:09.870436 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:09.870436 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:09.870436 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:09.870436 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:09.870436 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:09.871366 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:09.871366 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:09.871366 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:09.871366 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:09.871366 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:09.871366 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:09.871366 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:09.871366 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:09.871366 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:09.871366 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:09.871366 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 00:49:09.871366 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:49:09.871366 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 04 00:49:09.871366 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:09.876249 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 458/921] 10.222.0.27 () {56 vars in 1114 bytes} [Sun Sep 4 00:49:09 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:49:09.877526 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0eea8-84b8-4949-9f7b-dc3d6818750b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:09.889549 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-18d6397d-fd5a-4d4e-b34f-0f0539b73f63 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 04 00:49:09.889549 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:09.889549 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:09.889549 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:09.889549 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:09.889549 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:09.889549 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:09.889549 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:09.889549 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:09.889549 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:09.889549 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:09.889549 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:09.889549 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:49:09.889549 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:49:09.889549 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:49:09.889549 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:49:09.898046 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:09.898046 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:09.898046 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:49:09.898046 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:49:09.898046 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:49:09.898046 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:49:09.898046 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:09.898046 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:09.898046 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:09.898046 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:09.898046 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:09.898046 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:09.898046 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:09.898046 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:09.898046 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:09.898046 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:09.898046 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:09.898046 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:09.898992 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:09.898992 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:09.898992 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:49:09.898992 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:49:09.898992 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:49:09.898992 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:49:09.898992 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:49:09.898992 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:49:09.898992 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 04 00:49:09.898992 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:09.898992 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 464/922] 10.222.0.27 () {56 vars in 1117 bytes} [Sun Sep 4 00:49:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:09.901158 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d6397d-fd5a-4d4e-b34f-0f0539b73f63 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:09.982687 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0eea8-84b8-4949-9f7b-dc3d6818750b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:10.018877 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ee32e-1305-47fb-a3d6-1cc8935eeccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:10.019512 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ee32e-1305-47fb-a3d6-1cc8935eeccb None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:10.019793 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ee32e-1305-47fb-a3d6-1cc8935eeccb None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:10.020068 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ee32e-1305-47fb-a3d6-1cc8935eeccb None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:10.020499 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d6397d-fd5a-4d4e-b34f-0f0539b73f63 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:10.020726 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517ee32e-1305-47fb-a3d6-1cc8935eeccb None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:10.021036 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517ee32e-1305-47fb-a3d6-1cc8935eeccb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:10.022385 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517ee32e-1305-47fb-a3d6-1cc8935eeccb None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:10.032156 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 459/923] 10.222.0.27 () {56 vars in 1114 bytes} [Sun Sep 4 00:49:09 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:10.033507 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ee32e-1305-47fb-a3d6-1cc8935eeccb None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:10.041837 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:10.058217 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:10.062105 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b117f6f-b896-458b-b741-a3b64f2da6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:10.063842 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b117f6f-b896-458b-b741-a3b64f2da6fb None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:10.064358 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b117f6f-b896-458b-b741-a3b64f2da6fb None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:10.064842 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b117f6f-b896-458b-b741-a3b64f2da6fb None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:10.065766 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b117f6f-b896-458b-b741-a3b64f2da6fb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:10.066254 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b117f6f-b896-458b-b741-a3b64f2da6fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:10.067933 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b117f6f-b896-458b-b741-a3b64f2da6fb None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:10.092602 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 465/924] 10.222.0.27 () {56 vars in 1149 bytes} [Sun Sep 4 00:49:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:10.094589 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b117f6f-b896-458b-b741-a3b64f2da6fb None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:10.145628 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ee32e-1305-47fb-a3d6-1cc8935eeccb None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:10.182385 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc3efc-8550-4667-a566-a9dc04524184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:10.183103 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc3efc-8550-4667-a566-a9dc04524184 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:10.183430 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc3efc-8550-4667-a566-a9dc04524184 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:10.183705 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc3efc-8550-4667-a566-a9dc04524184 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:10.184323 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdc3efc-8550-4667-a566-a9dc04524184 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:10.184649 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdc3efc-8550-4667-a566-a9dc04524184 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:10.186213 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdc3efc-8550-4667-a566-a9dc04524184 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:10.192016 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 460/925] 10.222.0.27 () {56 vars in 1114 bytes} [Sun Sep 4 00:49:10 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:10.193244 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43174745-dba0-40de-942f-e028a470fb8d None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:10.193531 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43174745-dba0-40de-942f-e028a470fb8d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:10.193806 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43174745-dba0-40de-942f-e028a470fb8d None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:10.194992 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 461/926] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:49:10.196004 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0148d36-d012-4096-b040-8028c94e2a21 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:10.196285 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0148d36-d012-4096-b040-8028c94e2a21 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:10.196665 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0148d36-d012-4096-b040-8028c94e2a21 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:10.197759 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 462/927] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:49:10.198442 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0148d36-d012-4096-b040-8028c94e2a21 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:10.202510 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b117f6f-b896-458b-b741-a3b64f2da6fb None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:10.242333 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb249c28-113f-4b4f-ab04-233445b1881c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:10.242963 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb249c28-113f-4b4f-ab04-233445b1881c None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:10.243244 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb249c28-113f-4b4f-ab04-233445b1881c None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:10.243524 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb249c28-113f-4b4f-ab04-233445b1881c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:10.254072 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb249c28-113f-4b4f-ab04-233445b1881c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:10.255513 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb249c28-113f-4b4f-ab04-233445b1881c None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:10.270100 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-cb249c28-113f-4b4f-ab04-233445b1881c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:49:10.270100 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:10.270100 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:10.270100 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:10.270100 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:10.270100 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:10.270100 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:10.270100 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:10.270100 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:10.270100 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:10.270100 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:10.270100 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:10.270100 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:49:10.270100 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:49:10.270100 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:49:10.270100 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:49:10.271238 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:10.271238 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:10.271238 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:10.271238 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:10.271238 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:10.271238 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:10.271238 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:10.271238 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:10.271238 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:10.271238 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:10.271238 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:10.271238 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:10.271238 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:10.271238 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:10.271238 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:10.271238 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:10.271238 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:49:10.272322 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:49:10.272322 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:49:10.272322 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:10.272322 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 466/928] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:49:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:49:10.272780 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0bed6a-68d6-4f09-be1e-d6c0ebda8dfc None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:10.273065 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0bed6a-68d6-4f09-be1e-d6c0ebda8dfc None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:10.273341 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0bed6a-68d6-4f09-be1e-d6c0ebda8dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:10.286605 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:10.291118 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-ac0bed6a-68d6-4f09-be1e-d6c0ebda8dfc None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:10.292312 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ac0bed6a-68d6-4f09-be1e-d6c0ebda8dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:10.303415 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0148d36-d012-4096-b040-8028c94e2a21 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:10.318030 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:10.337298 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f571f-522b-42a9-ba46-f6262771ea83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:10.337920 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f571f-522b-42a9-ba46-f6262771ea83 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:10.338391 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f571f-522b-42a9-ba46-f6262771ea83 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:10.338837 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f571f-522b-42a9-ba46-f6262771ea83 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:10.339867 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896f571f-522b-42a9-ba46-f6262771ea83 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:10.341260 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896f571f-522b-42a9-ba46-f6262771ea83 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:10.358560 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 467/929] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:49:10.360672 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b18574-3533-47a1-909c-6b600f30aed3 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:10.360959 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b18574-3533-47a1-909c-6b600f30aed3 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:10.361237 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b18574-3533-47a1-909c-6b600f30aed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:10.379628 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-25b18574-3533-47a1-909c-6b600f30aed3 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:10.380860 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-25b18574-3533-47a1-909c-6b600f30aed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:10.448069 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 468/930] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:49:10.448782 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b18574-3533-47a1-909c-6b600f30aed3 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:10.451337 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 463/931] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 00:49:10 2022] POST /identity/v3/endpoints => generated 346 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:49:10.561887 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:10.577726 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b18574-3533-47a1-909c-6b600f30aed3 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:10.627173 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62515adf-8207-44bf-afa5-2f62930608de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:10.627850 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62515adf-8207-44bf-afa5-2f62930608de None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:10.628134 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62515adf-8207-44bf-afa5-2f62930608de None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:10.628408 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62515adf-8207-44bf-afa5-2f62930608de None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:10.629106 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62515adf-8207-44bf-afa5-2f62930608de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:10.629408 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62515adf-8207-44bf-afa5-2f62930608de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:10.630791 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62515adf-8207-44bf-afa5-2f62930608de None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:10.634788 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f571f-522b-42a9-ba46-f6262771ea83 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:10.648851 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 469/932] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:49:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:10.654459 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62515adf-8207-44bf-afa5-2f62930608de None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:10.761445 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f571f-522b-42a9-ba46-f6262771ea83 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:10.773976 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62515adf-8207-44bf-afa5-2f62930608de None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:10.803961 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b274e-8caf-402c-8dcc-03cb75430976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:10.804961 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b274e-8caf-402c-8dcc-03cb75430976 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:10.805243 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b274e-8caf-402c-8dcc-03cb75430976 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:10.805518 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b274e-8caf-402c-8dcc-03cb75430976 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:10.806099 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138b274e-8caf-402c-8dcc-03cb75430976 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:10.807461 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138b274e-8caf-402c-8dcc-03cb75430976 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:10.813956 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2660199-6f5b-46d0-b31a-0cb9094ff82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:10.814768 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2660199-6f5b-46d0-b31a-0cb9094ff82e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:10.815190 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2660199-6f5b-46d0-b31a-0cb9094ff82e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:10.815615 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2660199-6f5b-46d0-b31a-0cb9094ff82e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:10.816384 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-138b274e-8caf-402c-8dcc-03cb75430976 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 04 00:49:10.816384 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:10.816384 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 00:49:10.816384 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 00:49:10.816384 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 00:49:10.816384 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 00:49:10.816384 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 00:49:10.816384 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:49:10.816384 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 04 00:49:10.816384 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:10.816384 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 00:49:10.816384 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:10.816384 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:10.816384 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:10.816384 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:10.816384 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:10.816384 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:10.817583 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:10.817583 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:10.817583 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:10.817583 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:10.817583 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:10.817583 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:10.817583 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 00:49:10.817583 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 00:49:10.817583 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 00:49:10.817583 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 00:49:10.817583 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:10.817583 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:10.817583 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:10.817583 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:10.817583 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:10.817583 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:10.818486 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:10.818486 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:10.818486 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:10.818486 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:10.818486 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:10.818486 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:10.818486 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:10.818486 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:10.818486 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:10.818486 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:10.818486 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 00:49:10.818486 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:49:10.818486 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 04 00:49:10.818486 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:10.818486 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 464/933] 10.222.0.27 () {56 vars in 1126 bytes} [Sun Sep 4 00:49:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:49:10.819287 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b274e-8caf-402c-8dcc-03cb75430976 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:10.826251 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:10.829056 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2660199-6f5b-46d0-b31a-0cb9094ff82e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:10.831127 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2660199-6f5b-46d0-b31a-0cb9094ff82e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:10.846854 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-f2660199-6f5b-46d0-b31a-0cb9094ff82e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:49:10.846854 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:10.846854 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:10.846854 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:10.846854 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:10.846854 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:10.846854 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:10.846854 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:10.846854 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:10.846854 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:10.846854 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:10.846854 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:10.846854 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:49:10.846854 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:49:10.846854 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:49:10.847794 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:49:10.847794 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:10.847794 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:10.847794 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:10.847794 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:10.847794 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:10.847794 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:10.847794 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:10.847794 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:10.847794 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:10.847794 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:10.847794 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:10.847794 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:10.847794 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:10.847794 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:10.847794 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:10.847794 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:10.848777 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:49:10.848777 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:49:10.848777 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:49:10.848777 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:49:10.848777 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:49:10.848777 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:49:10.848777 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:49:10.848777 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:10.852594 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 470/934] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:49:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:49:10.853413 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2660199-6f5b-46d0-b31a-0cb9094ff82e None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:10.857421 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:10.933948 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b274e-8caf-402c-8dcc-03cb75430976 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:10.957875 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2660199-6f5b-46d0-b31a-0cb9094ff82e None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:10.974956 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0f787-9419-4569-a812-067f76593fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:10.975725 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0f787-9419-4569-a812-067f76593fe4 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:10.975836 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0f787-9419-4569-a812-067f76593fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:10.976114 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0f787-9419-4569-a812-067f76593fe4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:10.976715 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0f787-9419-4569-a812-067f76593fe4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:10.978065 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0f787-9419-4569-a812-067f76593fe4 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:10.987686 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-e7e0f787-9419-4569-a812-067f76593fe4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 04 00:49:10.987686 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:10.987686 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 00:49:10.987686 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 00:49:10.987686 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 00:49:10.987686 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 00:49:10.987686 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 00:49:10.987686 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:49:10.987686 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 04 00:49:10.987686 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:10.987686 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 00:49:10.987686 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:10.987686 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:10.987686 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:10.987686 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:10.987686 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:10.988678 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:10.988678 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:10.988678 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:10.988678 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:10.988678 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:10.988678 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:10.988678 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:10.988678 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 00:49:10.988678 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 00:49:10.988678 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 00:49:10.988678 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 00:49:10.988678 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:10.988678 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:10.988678 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:10.988678 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:10.988678 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:10.988678 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:10.993496 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:10.993496 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:10.993496 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:10.993496 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:10.993496 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:10.993496 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:10.993496 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:10.993496 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:10.993496 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:10.993496 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:10.993496 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 00:49:10.993496 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:49:10.993496 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 04 00:49:10.993496 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:10.993496 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 465/935] 10.222.0.27 () {56 vars in 1132 bytes} [Sun Sep 4 00:49:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:10.994260 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0f787-9419-4569-a812-067f76593fe4 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:11.001070 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244bbc1-0992-4454-a4a1-e24d52d43c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:11.001682 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244bbc1-0992-4454-a4a1-e24d52d43c94 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:11.001973 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244bbc1-0992-4454-a4a1-e24d52d43c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:11.002229 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244bbc1-0992-4454-a4a1-e24d52d43c94 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:11.002862 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244bbc1-0992-4454-a4a1-e24d52d43c94 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:11.003161 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244bbc1-0992-4454-a4a1-e24d52d43c94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:11.004570 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244bbc1-0992-4454-a4a1-e24d52d43c94 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:11.009289 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 471/936] 10.222.0.27 () {56 vars in 1122 bytes} [Sun Sep 4 00:49:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:11.010061 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244bbc1-0992-4454-a4a1-e24d52d43c94 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:11.072410 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:11.086565 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:11.107765 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0f787-9419-4569-a812-067f76593fe4 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:11.123588 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244bbc1-0992-4454-a4a1-e24d52d43c94 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:11.147095 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4063236-9d8e-443c-b8d9-33a40aafcae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:11.147701 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4063236-9d8e-443c-b8d9-33a40aafcae5 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:11.148135 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4063236-9d8e-443c-b8d9-33a40aafcae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:11.148410 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4063236-9d8e-443c-b8d9-33a40aafcae5 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:11.149433 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4063236-9d8e-443c-b8d9-33a40aafcae5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:11.150009 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4063236-9d8e-443c-b8d9-33a40aafcae5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:11.151880 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4063236-9d8e-443c-b8d9-33a40aafcae5 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:11.159838 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f82b5-9b3e-4b8f-a3f0-9fc8c32a9f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:11.161097 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f82b5-9b3e-4b8f-a3f0-9fc8c32a9f59 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:11.161377 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f82b5-9b3e-4b8f-a3f0-9fc8c32a9f59 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:11.161650 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f82b5-9b3e-4b8f-a3f0-9fc8c32a9f59 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:11.162763 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f82b5-9b3e-4b8f-a3f0-9fc8c32a9f59 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:11.163123 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f82b5-9b3e-4b8f-a3f0-9fc8c32a9f59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:11.164707 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f82b5-9b3e-4b8f-a3f0-9fc8c32a9f59 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:11.170543 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 466/937] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:49:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:11.170543 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4063236-9d8e-443c-b8d9-33a40aafcae5 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:11.179774 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 472/938] 10.222.0.27 () {56 vars in 1126 bytes} [Sun Sep 4 00:49:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:11.179774 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f82b5-9b3e-4b8f-a3f0-9fc8c32a9f59 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:11.277366 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4063236-9d8e-443c-b8d9-33a40aafcae5 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:11.291246 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f82b5-9b3e-4b8f-a3f0-9fc8c32a9f59 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:11.305741 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:11.314203 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b91101-e115-4e28-ac6f-1ad63e38a1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:11.314807 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b91101-e115-4e28-ac6f-1ad63e38a1bd None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:11.315080 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b91101-e115-4e28-ac6f-1ad63e38a1bd None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:11.315350 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b91101-e115-4e28-ac6f-1ad63e38a1bd None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:11.315972 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b91101-e115-4e28-ac6f-1ad63e38a1bd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:11.316270 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b91101-e115-4e28-ac6f-1ad63e38a1bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:11.317655 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b91101-e115-4e28-ac6f-1ad63e38a1bd None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:11.325578 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:11.327273 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 467/939] 10.222.0.27 () {56 vars in 1122 bytes} [Sun Sep 4 00:49:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:11.328871 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b91101-e115-4e28-ac6f-1ad63e38a1bd None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:11.330967 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead44b99-27b8-4d94-9672-b24da56c6e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:11.331705 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead44b99-27b8-4d94-9672-b24da56c6e0d None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:11.331985 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead44b99-27b8-4d94-9672-b24da56c6e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:11.332257 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead44b99-27b8-4d94-9672-b24da56c6e0d None admin] PATH_INFO: `/v3/projects/5100a8d316d243fb8f82e9044e1b3b06/users/d6d01ca673a1404c9b4cea51036c595c/roles/91d340dbd1f142c5aa33ad640eca891c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:11.334425 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead44b99-27b8-4d94-9672-b24da56c6e0d None admin] RBAC: Authorizing `identity:create_grant(project_id=5100a8d316d243fb8f82e9044e1b3b06, user_id=d6d01ca673a1404c9b4cea51036c595c, role_id=91d340dbd1f142c5aa33ad640eca891c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:11.335854 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead44b99-27b8-4d94-9672-b24da56c6e0d None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:11.430478 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b91101-e115-4e28-ac6f-1ad63e38a1bd None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:11.439257 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 473/940] 10.222.0.27 () {58 vars in 1439 bytes} [Sun Sep 4 00:49:11 2022] PUT /identity/v3/projects/5100a8d316d243fb8f82e9044e1b3b06/users/d6d01ca673a1404c9b4cea51036c595c/roles/91d340dbd1f142c5aa33ad640eca891c => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 00:49:11.449720 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead44b99-27b8-4d94-9672-b24da56c6e0d None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:11.471201 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22552bd6-635a-4ae1-8961-888ff2cdd68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:11.471827 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22552bd6-635a-4ae1-8961-888ff2cdd68e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:11.472108 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22552bd6-635a-4ae1-8961-888ff2cdd68e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:11.472378 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22552bd6-635a-4ae1-8961-888ff2cdd68e None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:11.473457 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22552bd6-635a-4ae1-8961-888ff2cdd68e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:11.473457 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22552bd6-635a-4ae1-8961-888ff2cdd68e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:11.474794 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22552bd6-635a-4ae1-8961-888ff2cdd68e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:11.480956 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 468/941] 10.222.0.27 () {56 vars in 1126 bytes} [Sun Sep 4 00:49:11 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:11.486768 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22552bd6-635a-4ae1-8961-888ff2cdd68e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:11.558553 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:11.595683 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead44b99-27b8-4d94-9672-b24da56c6e0d None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:11.608839 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22552bd6-635a-4ae1-8961-888ff2cdd68e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:11.634156 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f81eb-93a0-4ba8-8297-10d556a4c962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:11.635046 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f81eb-93a0-4ba8-8297-10d556a4c962 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:11.635332 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f81eb-93a0-4ba8-8297-10d556a4c962 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:11.635607 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f81eb-93a0-4ba8-8297-10d556a4c962 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:11.636330 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0f81eb-93a0-4ba8-8297-10d556a4c962 None admin] RBAC: Adding query filter params (interface=public, service_id=b3d567d3ce664858b753ae51f5d61605, region_id=RegionOne) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:11.636694 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0f81eb-93a0-4ba8-8297-10d556a4c962 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:11.638116 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0f81eb-93a0-4ba8-8297-10d556a4c962 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:11.644511 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 474/942] 10.222.0.27 () {56 vars in 1251 bytes} [Sun Sep 4 00:49:11 2022] GET /identity/v3/endpoints?service_id=b3d567d3ce664858b753ae51f5d61605&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:11.649471 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad0e491-08ff-46b4-9d67-da55a94da36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:11.650111 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0e491-08ff-46b4-9d67-da55a94da36b None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:11.650395 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0e491-08ff-46b4-9d67-da55a94da36b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:11.650675 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0e491-08ff-46b4-9d67-da55a94da36b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:11.650837 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f81eb-93a0-4ba8-8297-10d556a4c962 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:11.651366 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad0e491-08ff-46b4-9d67-da55a94da36b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:11.652773 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad0e491-08ff-46b4-9d67-da55a94da36b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:11.713944 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:11.759642 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f81eb-93a0-4ba8-8297-10d556a4c962 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:11.776339 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 469/943] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 00:49:11 2022] POST /identity/v3/endpoints => generated 347 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:49:11.814178 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b753759b-8901-4d55-af2a-71e94699e80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:11.814884 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753759b-8901-4d55-af2a-71e94699e80d None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:11.815138 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753759b-8901-4d55-af2a-71e94699e80d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:11.815420 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753759b-8901-4d55-af2a-71e94699e80d None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:11.816029 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b753759b-8901-4d55-af2a-71e94699e80d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:11.817619 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b753759b-8901-4d55-af2a-71e94699e80d None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:11.825326 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 475/944] 10.222.0.27 () {56 vars in 1087 bytes} [Sun Sep 4 00:49:11 2022] GET /identity/v3/services => generated 1497 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 00:49:12.488443 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb3e13-1987-4f18-9612-0c7750754f0b None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:12.488814 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb3e13-1987-4f18-9612-0c7750754f0b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:12.488935 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb3e13-1987-4f18-9612-0c7750754f0b None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:12.490523 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 470/945] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:49:12.495629 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8618610-d187-4a8d-8839-5cd691b1ffee None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:12.495918 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8618610-d187-4a8d-8839-5cd691b1ffee None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:12.496192 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8618610-d187-4a8d-8839-5cd691b1ffee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:12.515271 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-c8618610-d187-4a8d-8839-5cd691b1ffee None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:12.517022 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c8618610-d187-4a8d-8839-5cd691b1ffee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:12.586201 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 476/946] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:49:12.855921 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb3e13-1987-4f18-9612-0c7750754f0b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:12.970422 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb3e13-1987-4f18-9612-0c7750754f0b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:12.984170 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:13.005690 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14d47a1-7e99-40e9-adfa-991e76eddd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:13.006297 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d47a1-7e99-40e9-adfa-991e76eddd88 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:13.006574 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d47a1-7e99-40e9-adfa-991e76eddd88 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:13.006844 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d47a1-7e99-40e9-adfa-991e76eddd88 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:13.007491 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14d47a1-7e99-40e9-adfa-991e76eddd88 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:13.008925 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14d47a1-7e99-40e9-adfa-991e76eddd88 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:13.011529 dv-cinder-master-3-smb devstack@keystone.service[81629]: WARNING keystone.server.flask.common [None req-b14d47a1-7e99-40e9-adfa-991e76eddd88 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 04 00:49:13.054991 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.sql.core [None req-b14d47a1-7e99-40e9-adfa-991e76eddd88 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 04 00:49:13.054991 dv-cinder-master-3-smb devstack@keystone.service[81629]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 04 00:49:13.054991 dv-cinder-master-3-smb devstack@keystone.service[81629]: [parameters: {'id': '31002d844e1947daa2207cc78ced9e4c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 04 00:49:13.054991 dv-cinder-master-3-smb devstack@keystone.service[81629]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81629) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 04 00:49:13.072012 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-b14d47a1-7e99-40e9-adfa-991e76eddd88 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 31002d844e1947daa2207cc78ced9e4c.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 31002d844e1947daa2207cc78ced9e4c. Sep 04 00:49:13.072012 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:13.072012 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 04 00:49:13.072012 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 04 00:49:13.072012 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 04 00:49:13.072012 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 04 00:49:13.072012 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 04 00:49:13.072012 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application result = self._query(query) Sep 04 00:49:13.072012 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 04 00:49:13.072012 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application conn.query(q) Sep 04 00:49:13.072012 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 04 00:49:13.072012 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 04 00:49:13.072012 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 04 00:49:13.073893 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application result.read() Sep 04 00:49:13.073893 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 04 00:49:13.073893 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 04 00:49:13.073893 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 04 00:49:13.073893 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 04 00:49:13.073893 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 04 00:49:13.073893 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 04 00:49:13.073893 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 04 00:49:13.073893 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 04 00:49:13.073893 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 04 00:49:13.073893 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:13.073893 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 04 00:49:13.073893 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:13.073893 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:13.073893 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 04 00:49:13.073893 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 04 00:49:13.073893 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 04 00:49:13.073893 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 04 00:49:13.074929 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 04 00:49:13.074929 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application next(self.gen) Sep 04 00:49:13.074929 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 04 00:49:13.074929 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application yield resource Sep 04 00:49:13.074929 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 04 00:49:13.074929 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application next(self.gen) Sep 04 00:49:13.074929 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 04 00:49:13.074929 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self.session.rollback() Sep 04 00:49:13.074929 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 04 00:49:13.074929 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self.force_reraise() Sep 04 00:49:13.074929 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 04 00:49:13.074929 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise self.value Sep 04 00:49:13.074929 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 04 00:49:13.074929 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 04 00:49:13.074929 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 04 00:49:13.074929 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application session.commit() Sep 04 00:49:13.074929 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 04 00:49:13.075839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 04 00:49:13.075839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 04 00:49:13.075839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._prepare_impl() Sep 04 00:49:13.075839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 04 00:49:13.075839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self.session.flush() Sep 04 00:49:13.075839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 04 00:49:13.075839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._flush(objects) Sep 04 00:49:13.075839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 04 00:49:13.075839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 04 00:49:13.075839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 04 00:49:13.075839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application compat.raise_( Sep 04 00:49:13.075839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 04 00:49:13.075839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception Sep 04 00:49:13.075839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 04 00:49:13.075839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application flush_context.execute() Sep 04 00:49:13.075839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 04 00:49:13.075839 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rec.execute(self) Sep 04 00:49:13.076776 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 04 00:49:13.076776 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 04 00:49:13.076776 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 04 00:49:13.076776 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 04 00:49:13.076776 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 04 00:49:13.076776 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 04 00:49:13.076776 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 04 00:49:13.076776 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 04 00:49:13.076776 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 04 00:49:13.076776 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 04 00:49:13.076776 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 04 00:49:13.076776 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 04 00:49:13.076776 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 04 00:49:13.076776 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 04 00:49:13.076776 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 04 00:49:13.077577 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 04 00:49:13.077577 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 04 00:49:13.077577 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception Sep 04 00:49:13.077577 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 04 00:49:13.077577 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 04 00:49:13.077577 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 04 00:49:13.077577 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 04 00:49:13.077577 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 04 00:49:13.077577 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application result = self._query(query) Sep 04 00:49:13.077577 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 04 00:49:13.077577 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application conn.query(q) Sep 04 00:49:13.077577 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 04 00:49:13.077577 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 04 00:49:13.077577 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 04 00:49:13.077577 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application result.read() Sep 04 00:49:13.077577 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 04 00:49:13.077577 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 04 00:49:13.080361 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 04 00:49:13.080361 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 04 00:49:13.080361 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 04 00:49:13.080361 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 04 00:49:13.080361 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 04 00:49:13.080361 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 04 00:49:13.080361 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 04 00:49:13.080361 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 04 00:49:13.080361 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [parameters: {'id': '31002d844e1947daa2207cc78ced9e4c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 04 00:49:13.080361 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 04 00:49:13.080361 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:13.080361 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 00:49:13.080361 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:13.080361 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:13.080361 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 04 00:49:13.082159 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 04 00:49:13.082159 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 04 00:49:13.082159 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 04 00:49:13.082159 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 04 00:49:13.082159 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:13.082159 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 00:49:13.082159 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:13.082159 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:13.082159 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:13.082159 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:13.082159 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:13.082159 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:13.082159 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:13.082159 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:13.082159 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:13.082159 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:13.082159 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:13.090070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:13.090070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 04 00:49:13.090070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 04 00:49:13.090070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:13.090070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:13.090070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 04 00:49:13.090070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 04 00:49:13.090070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 31002d844e1947daa2207cc78ced9e4c. Sep 04 00:49:13.090070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:13.090070 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 471/947] 10.222.0.27 () {60 vars in 1140 bytes} [Sun Sep 4 00:49:12 2022] POST /identity/v3/projects => generated 262 bytes in 218 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:13.090751 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8618610-d187-4a8d-8839-5cd691b1ffee None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:13.130228 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:13.185654 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8618610-d187-4a8d-8839-5cd691b1ffee None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:13.223592 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f888a4fa-0fb5-4238-b5ab-5c85513bf527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:13.224237 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888a4fa-0fb5-4238-b5ab-5c85513bf527 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:13.224556 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888a4fa-0fb5-4238-b5ab-5c85513bf527 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:13.224831 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888a4fa-0fb5-4238-b5ab-5c85513bf527 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:13.232151 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f888a4fa-0fb5-4238-b5ab-5c85513bf527 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:13.233960 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f888a4fa-0fb5-4238-b5ab-5c85513bf527 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:13.246015 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-f888a4fa-0fb5-4238-b5ab-5c85513bf527 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:49:13.246015 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:13.246015 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:13.246015 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:13.246015 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:13.246015 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:13.246015 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:13.246015 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:13.246015 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:13.246015 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:13.246015 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:13.246015 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:13.246015 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:49:13.246015 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:49:13.246015 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:49:13.248157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:49:13.248157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:13.248157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:13.248157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:13.248157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:13.248157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:13.248157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:13.248157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:13.248157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:13.248157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:13.248157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:13.248157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:13.248157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:13.248157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:13.248157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:13.248157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:13.248157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:13.250868 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:49:13.250868 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:49:13.250868 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:49:13.250868 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:49:13.250868 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:49:13.250868 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:49:13.250868 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:49:13.250868 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:13.250868 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 477/948] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:49:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:13.254440 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14d47a1-7e99-40e9-adfa-991e76eddd88 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:13.360301 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14d47a1-7e99-40e9-adfa-991e76eddd88 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:13.397226 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d29c59-7914-4d17-8e08-27e56fdc2af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:13.397756 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d29c59-7914-4d17-8e08-27e56fdc2af4 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:13.398073 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d29c59-7914-4d17-8e08-27e56fdc2af4 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:13.398352 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d29c59-7914-4d17-8e08-27e56fdc2af4 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:13.399009 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d29c59-7914-4d17-8e08-27e56fdc2af4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:13.399310 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d29c59-7914-4d17-8e08-27e56fdc2af4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:13.400976 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d29c59-7914-4d17-8e08-27e56fdc2af4 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:13.425330 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 472/949] 10.222.0.27 () {56 vars in 1112 bytes} [Sun Sep 4 00:49:13 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:13.487083 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1c7ca-1ccc-4060-b44a-f2e4d88f92f5 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:13.487500 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1c7ca-1ccc-4060-b44a-f2e4d88f92f5 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:13.487880 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1c7ca-1ccc-4060-b44a-f2e4d88f92f5 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:13.490128 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 478/950] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:49:13.496377 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c3b85-0285-4cb3-8219-39e35b72330c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:13.496698 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c3b85-0285-4cb3-8219-39e35b72330c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:13.496977 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c3b85-0285-4cb3-8219-39e35b72330c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:13.518156 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-9e7c3b85-0285-4cb3-8219-39e35b72330c None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:13.519740 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9e7c3b85-0285-4cb3-8219-39e35b72330c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:13.547525 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:13.598854 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 473/951] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:49:13.814966 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbfb79-102a-4bab-a540-06f27af87b04 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:13.815340 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbfb79-102a-4bab-a540-06f27af87b04 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:13.815518 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbfb79-102a-4bab-a540-06f27af87b04 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:13.817165 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 479/952] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:49:13.822385 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91332ddd-ef74-469e-9e3a-73da318eca5f None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:13.822726 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91332ddd-ef74-469e-9e3a-73da318eca5f None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:13.822956 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91332ddd-ef74-469e-9e3a-73da318eca5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:13.842238 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-91332ddd-ef74-469e-9e3a-73da318eca5f None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:13.843445 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-91332ddd-ef74-469e-9e3a-73da318eca5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:13.879600 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cbfb79-102a-4bab-a540-06f27af87b04 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:13.926393 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 474/953] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:49:13.927750 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef896bcf-9da9-4bee-a748-fcbbe867c0ec None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:13.928037 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef896bcf-9da9-4bee-a748-fcbbe867c0ec None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:13.928352 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef896bcf-9da9-4bee-a748-fcbbe867c0ec None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:13.933520 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 475/954] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:49:13.936558 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df1c79-c5c8-4cd9-a336-16cbfce2bd51 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:13.936841 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df1c79-c5c8-4cd9-a336-16cbfce2bd51 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:13.937113 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df1c79-c5c8-4cd9-a336-16cbfce2bd51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:13.958383 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-52df1c79-c5c8-4cd9-a336-16cbfce2bd51 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:13.959779 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-52df1c79-c5c8-4cd9-a336-16cbfce2bd51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:14.003029 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cbfb79-102a-4bab-a540-06f27af87b04 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:14.010775 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:14.040630 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 476/955] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 00:49:14.049233 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0818d-4a88-45d9-95a9-0a58f12052d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:14.050223 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0818d-4a88-45d9-95a9-0a58f12052d6 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:14.050711 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0818d-4a88-45d9-95a9-0a58f12052d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:14.051214 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0818d-4a88-45d9-95a9-0a58f12052d6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:14.063061 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef0818d-4a88-45d9-95a9-0a58f12052d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:49:14.068414 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-9ef0818d-4a88-45d9-95a9-0a58f12052d6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 04 00:49:14.068414 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:14.068414 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:14.068414 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:14.068414 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:14.068414 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:14.068414 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:14.068414 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:14.068414 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:14.068414 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:14.068414 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:14.068414 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:14.068414 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:49:14.068414 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:49:14.068414 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:49:14.068414 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:49:14.069437 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:49:14.069437 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:49:14.069437 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:49:14.069437 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:49:14.069437 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:14.069437 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:14.069437 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:14.069437 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:14.069437 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:14.069437 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:14.069437 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:14.069437 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:14.069437 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:14.069437 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:14.069437 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:14.069437 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:14.070344 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:14.070344 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:14.070344 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:14.070344 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:14.070344 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:49:14.070344 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:49:14.070344 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:49:14.070344 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:49:14.070344 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:49:14.070344 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:49:14.070344 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 04 00:49:14.070344 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:14.073079 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 480/956] 10.222.0.27 () {56 vars in 1120 bytes} [Sun Sep 4 00:49:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:49:14.080271 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df1c79-c5c8-4cd9-a336-16cbfce2bd51 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:14.101150 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:14.189564 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df1c79-c5c8-4cd9-a336-16cbfce2bd51 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:14.224662 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5979abc-e235-4662-85d2-d6607e71d297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:14.225512 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5979abc-e235-4662-85d2-d6607e71d297 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:14.225981 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5979abc-e235-4662-85d2-d6607e71d297 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:14.226415 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5979abc-e235-4662-85d2-d6607e71d297 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:14.227302 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5979abc-e235-4662-85d2-d6607e71d297 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:14.227765 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5979abc-e235-4662-85d2-d6607e71d297 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:14.229314 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5979abc-e235-4662-85d2-d6607e71d297 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:14.236482 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0818d-4a88-45d9-95a9-0a58f12052d6 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:14.246882 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 477/957] 10.222.0.27 () {56 vars in 1115 bytes} [Sun Sep 4 00:49:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:14.252577 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5979abc-e235-4662-85d2-d6607e71d297 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:14.336510 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:14.342432 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0818d-4a88-45d9-95a9-0a58f12052d6 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:14.357325 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5979abc-e235-4662-85d2-d6607e71d297 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:14.378331 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3e862-da51-4bbf-a339-e6c2ad9cbcaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:14.378962 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3e862-da51-4bbf-a339-e6c2ad9cbcaf None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:14.379242 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3e862-da51-4bbf-a339-e6c2ad9cbcaf None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:14.379516 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3e862-da51-4bbf-a339-e6c2ad9cbcaf None admin] PATH_INFO: `/v3/services/compute` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:14.380130 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce3e862-da51-4bbf-a339-e6c2ad9cbcaf None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:14.381532 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce3e862-da51-4bbf-a339-e6c2ad9cbcaf None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:14.390494 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-3ce3e862-da51-4bbf-a339-e6c2ad9cbcaf None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 04 00:49:14.390494 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:14.390494 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 00:49:14.390494 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 00:49:14.390494 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 00:49:14.390494 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 00:49:14.390494 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 00:49:14.390494 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:49:14.390494 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 04 00:49:14.390494 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:14.390494 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 00:49:14.390494 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:14.390494 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:14.390494 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:14.390494 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:14.390494 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:14.390494 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:14.391586 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:14.391586 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:14.391586 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:14.391586 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:14.391586 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:14.391586 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:14.391586 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 00:49:14.391586 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 00:49:14.391586 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 00:49:14.391586 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 00:49:14.391586 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:14.391586 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:14.391586 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:14.391586 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:14.391586 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:14.391586 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:14.393036 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:14.393036 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:14.393036 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:14.393036 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:14.393036 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:14.393036 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:14.393036 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:14.393036 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:14.393036 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:14.393036 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:14.393036 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 00:49:14.393036 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:49:14.393036 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 04 00:49:14.393036 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:14.393036 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 481/958] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:49:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:14.393852 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3e862-da51-4bbf-a339-e6c2ad9cbcaf None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:14.400575 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374da0e3-cbd5-4716-900f-55ae8d277989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:14.401221 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374da0e3-cbd5-4716-900f-55ae8d277989 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:14.401506 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374da0e3-cbd5-4716-900f-55ae8d277989 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:14.401785 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374da0e3-cbd5-4716-900f-55ae8d277989 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:14.412948 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374da0e3-cbd5-4716-900f-55ae8d277989 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:14.414672 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374da0e3-cbd5-4716-900f-55ae8d277989 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:14.425661 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:14.431070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-374da0e3-cbd5-4716-900f-55ae8d277989 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:49:14.431070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:14.431070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:14.431070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:14.431070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:14.431070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:14.431070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:14.431070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:14.431070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:14.431070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:14.431070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:14.431070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:14.431070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:49:14.431070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:49:14.431070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:49:14.431070 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:49:14.432018 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:14.432018 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:14.432018 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:14.432018 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:14.432018 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:14.432018 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:14.432018 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:14.432018 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:14.432018 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:14.432018 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:14.432018 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:14.432018 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:14.432018 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:14.432018 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:14.432018 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:14.432018 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:14.432018 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:49:14.440731 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:49:14.440731 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:49:14.440731 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:14.440731 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 478/959] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:49:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:49:14.440731 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374da0e3-cbd5-4716-900f-55ae8d277989 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:14.502655 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3e862-da51-4bbf-a339-e6c2ad9cbcaf None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:14.539387 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a7e06-2ea6-4d9b-b439-a2cca8aaf381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:14.539932 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a7e06-2ea6-4d9b-b439-a2cca8aaf381 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:14.540213 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a7e06-2ea6-4d9b-b439-a2cca8aaf381 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:14.540503 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a7e06-2ea6-4d9b-b439-a2cca8aaf381 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:14.541085 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a7e06-2ea6-4d9b-b439-a2cca8aaf381 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:14.541661 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374da0e3-cbd5-4716-900f-55ae8d277989 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:14.542449 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a7e06-2ea6-4d9b-b439-a2cca8aaf381 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:14.552574 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-124a7e06-2ea6-4d9b-b439-a2cca8aaf381 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 04 00:49:14.552574 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:14.552574 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 00:49:14.552574 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 00:49:14.552574 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 00:49:14.552574 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 00:49:14.552574 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 00:49:14.552574 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:49:14.552574 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 04 00:49:14.552574 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:14.552574 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 00:49:14.552574 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:14.552574 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:14.552574 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:14.552574 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:14.552574 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:14.552574 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:14.553623 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:14.553623 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:14.553623 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:14.553623 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:14.553623 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:14.553623 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:14.553623 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 00:49:14.553623 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 00:49:14.553623 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 00:49:14.553623 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 00:49:14.553623 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:14.553623 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:14.553623 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:14.553623 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:14.553623 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:14.553623 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:14.554454 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:14.554454 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:14.554454 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:14.554454 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:14.554454 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:14.554454 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:14.554454 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:14.554454 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:14.554454 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:14.554454 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:14.554454 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 00:49:14.554454 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:49:14.554454 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 04 00:49:14.554454 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:14.554454 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 482/960] 10.222.0.27 () {56 vars in 1126 bytes} [Sun Sep 4 00:49:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:14.555748 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a7e06-2ea6-4d9b-b439-a2cca8aaf381 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:14.576912 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:14.581881 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e86a9df-221c-430f-9ee3-9a6f6866f9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:14.582508 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e86a9df-221c-430f-9ee3-9a6f6866f9c9 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:14.582785 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e86a9df-221c-430f-9ee3-9a6f6866f9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:14.583052 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e86a9df-221c-430f-9ee3-9a6f6866f9c9 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:14.583674 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e86a9df-221c-430f-9ee3-9a6f6866f9c9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:14.583973 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e86a9df-221c-430f-9ee3-9a6f6866f9c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:14.585388 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e86a9df-221c-430f-9ee3-9a6f6866f9c9 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:14.592140 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 479/961] 10.222.0.27 () {56 vars in 1112 bytes} [Sun Sep 4 00:49:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:14.592140 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e86a9df-221c-430f-9ee3-9a6f6866f9c9 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:14.672307 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:14.679660 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a7e06-2ea6-4d9b-b439-a2cca8aaf381 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:14.704735 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e86a9df-221c-430f-9ee3-9a6f6866f9c9 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:14.717631 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a4997-2e55-4211-a90e-97d1d6cd3bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:14.718240 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a4997-2e55-4211-a90e-97d1d6cd3bdb None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:14.718647 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a4997-2e55-4211-a90e-97d1d6cd3bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:14.718961 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a4997-2e55-4211-a90e-97d1d6cd3bdb None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:14.719606 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642a4997-2e55-4211-a90e-97d1d6cd3bdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:14.719980 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642a4997-2e55-4211-a90e-97d1d6cd3bdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:14.721419 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642a4997-2e55-4211-a90e-97d1d6cd3bdb None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:14.737874 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 483/962] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:14.737874 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a4997-2e55-4211-a90e-97d1d6cd3bdb None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:14.744726 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b6d30-08e5-4cfc-839d-9d36e596ab69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:14.745331 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b6d30-08e5-4cfc-839d-9d36e596ab69 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:14.745611 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b6d30-08e5-4cfc-839d-9d36e596ab69 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:14.745881 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b6d30-08e5-4cfc-839d-9d36e596ab69 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:14.746502 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20b6d30-08e5-4cfc-839d-9d36e596ab69 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:14.746810 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20b6d30-08e5-4cfc-839d-9d36e596ab69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:14.748136 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20b6d30-08e5-4cfc-839d-9d36e596ab69 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:14.753758 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 480/963] 10.222.0.27 () {56 vars in 1122 bytes} [Sun Sep 4 00:49:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:14.753758 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b6d30-08e5-4cfc-839d-9d36e596ab69 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:14.821269 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:14.849342 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a4997-2e55-4211-a90e-97d1d6cd3bdb None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:14.866793 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b6d30-08e5-4cfc-839d-9d36e596ab69 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:14.885641 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07017a-4249-4310-93cd-1a6696cc8415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:14.886220 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07017a-4249-4310-93cd-1a6696cc8415 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:14.892701 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07017a-4249-4310-93cd-1a6696cc8415 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:14.892701 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07017a-4249-4310-93cd-1a6696cc8415 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:14.892701 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f07017a-4249-4310-93cd-1a6696cc8415 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:14.892701 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f07017a-4249-4310-93cd-1a6696cc8415 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:14.892701 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f07017a-4249-4310-93cd-1a6696cc8415 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:14.894246 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 484/964] 10.222.0.27 () {56 vars in 1112 bytes} [Sun Sep 4 00:49:14 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:14.894868 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07017a-4249-4310-93cd-1a6696cc8415 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:14.908684 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076b779-ebcd-49a9-9716-a2968f0370cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:14.910200 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076b779-ebcd-49a9-9716-a2968f0370cf None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:14.911058 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076b779-ebcd-49a9-9716-a2968f0370cf None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:14.911881 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076b779-ebcd-49a9-9716-a2968f0370cf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:14.918787 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:14.926313 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7076b779-ebcd-49a9-9716-a2968f0370cf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:14.929323 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7076b779-ebcd-49a9-9716-a2968f0370cf None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:14.946892 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-7076b779-ebcd-49a9-9716-a2968f0370cf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 04 00:49:14.946892 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:14.946892 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:14.946892 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:14.946892 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:14.946892 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:14.946892 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:14.946892 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:14.946892 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:14.946892 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:14.946892 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:14.946892 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:14.946892 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:49:14.946892 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:49:14.946892 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:49:14.946892 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:49:14.954793 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:14.954793 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:14.954793 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:49:14.954793 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:49:14.954793 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:49:14.954793 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:49:14.954793 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:14.954793 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:14.954793 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:14.954793 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:14.954793 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:14.954793 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:14.954793 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:14.954793 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:14.954793 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:14.954793 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:14.954793 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:14.954793 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:14.955730 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:14.955730 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:14.955730 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:49:14.955730 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:49:14.955730 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:49:14.955730 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:49:14.955730 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:49:14.955730 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:49:14.955730 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 04 00:49:14.955730 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:14.955730 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 481/965] 10.222.0.27 () {56 vars in 1117 bytes} [Sun Sep 4 00:49:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:14.958413 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076b779-ebcd-49a9-9716-a2968f0370cf None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:15.026017 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07017a-4249-4310-93cd-1a6696cc8415 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:15.071710 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d5c5b-0f14-4df8-ae0a-dc3c006b7711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:15.072341 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d5c5b-0f14-4df8-ae0a-dc3c006b7711 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:15.072667 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d5c5b-0f14-4df8-ae0a-dc3c006b7711 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:15.074045 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d5c5b-0f14-4df8-ae0a-dc3c006b7711 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:15.074232 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d5c5b-0f14-4df8-ae0a-dc3c006b7711 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:15.074504 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d5c5b-0f14-4df8-ae0a-dc3c006b7711 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:15.076086 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d5c5b-0f14-4df8-ae0a-dc3c006b7711 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:15.084010 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 485/966] 10.222.0.27 () {56 vars in 1122 bytes} [Sun Sep 4 00:49:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:15.089590 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d5c5b-0f14-4df8-ae0a-dc3c006b7711 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:15.104819 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076b779-ebcd-49a9-9716-a2968f0370cf None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:15.128524 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:15.148740 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc04f65a-1dbc-4a31-9827-c9afb2fdd489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:15.149460 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04f65a-1dbc-4a31-9827-c9afb2fdd489 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:15.149749 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04f65a-1dbc-4a31-9827-c9afb2fdd489 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:15.150041 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04f65a-1dbc-4a31-9827-c9afb2fdd489 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:15.150794 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc04f65a-1dbc-4a31-9827-c9afb2fdd489 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:15.151102 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc04f65a-1dbc-4a31-9827-c9afb2fdd489 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:15.152743 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc04f65a-1dbc-4a31-9827-c9afb2fdd489 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:15.174672 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 482/967] 10.222.0.27 () {56 vars in 1149 bytes} [Sun Sep 4 00:49:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:15.180680 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc04f65a-1dbc-4a31-9827-c9afb2fdd489 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:15.190416 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:15.212763 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d5c5b-0f14-4df8-ae0a-dc3c006b7711 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:15.252474 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf03193-6974-4a67-b383-a64699d3f15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:15.253147 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf03193-6974-4a67-b383-a64699d3f15e None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:15.253250 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf03193-6974-4a67-b383-a64699d3f15e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:15.253480 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf03193-6974-4a67-b383-a64699d3f15e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:15.254163 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf03193-6974-4a67-b383-a64699d3f15e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:15.255535 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf03193-6974-4a67-b383-a64699d3f15e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:15.294391 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc04f65a-1dbc-4a31-9827-c9afb2fdd489 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:15.332569 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5998e-7d0f-4eca-8b58-3b5dd9655c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:15.333182 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5998e-7d0f-4eca-8b58-3b5dd9655c17 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:15.333468 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5998e-7d0f-4eca-8b58-3b5dd9655c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:15.333741 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5998e-7d0f-4eca-8b58-3b5dd9655c17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:15.345169 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 486/968] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 00:49:15 2022] POST /identity/v3/endpoints => generated 349 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 00:49:15.391244 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:15.416540 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5998e-7d0f-4eca-8b58-3b5dd9655c17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:15.418120 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5998e-7d0f-4eca-8b58-3b5dd9655c17 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:15.474487 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a4631-cd98-4dc7-8941-13272c0c625d None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:15.474844 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a4631-cd98-4dc7-8941-13272c0c625d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:15.475061 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a4631-cd98-4dc7-8941-13272c0c625d None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:15.476582 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 487/969] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 00:49:15.482109 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3840a4-1f71-433d-b413-a91fb731616d None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:15.482399 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3840a4-1f71-433d-b413-a91fb731616d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:15.482675 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3840a4-1f71-433d-b413-a91fb731616d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:15.506823 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-3a3840a4-1f71-433d-b413-a91fb731616d None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:15.508857 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3a3840a4-1f71-433d-b413-a91fb731616d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:15.552545 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 483/970] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:49:15 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 04 00:49:15.558652 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5998e-7d0f-4eca-8b58-3b5dd9655c17 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:15.604338 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 488/971] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 00:49:15.648886 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:15.671195 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5998e-7d0f-4eca-8b58-3b5dd9655c17 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:15.707637 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a8589-27f6-4a49-a259-808cfeb14881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:15.708365 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a8589-27f6-4a49-a259-808cfeb14881 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:15.708743 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a8589-27f6-4a49-a259-808cfeb14881 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:15.709093 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a8589-27f6-4a49-a259-808cfeb14881 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:15.720183 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a8589-27f6-4a49-a259-808cfeb14881 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:15.721775 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a8589-27f6-4a49-a259-808cfeb14881 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:15.736832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-867a8589-27f6-4a49-a259-808cfeb14881 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:49:15.736832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:15.736832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:15.736832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:15.736832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:15.736832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:15.736832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:15.736832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:15.736832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:15.736832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:15.736832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:15.736832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:15.736832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:49:15.736832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:49:15.736832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:49:15.736832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:49:15.737989 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:15.737989 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:15.737989 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:15.737989 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:15.737989 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:15.737989 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:15.737989 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:15.737989 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:15.737989 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:15.737989 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:15.737989 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:15.737989 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:15.737989 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:15.737989 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:15.737989 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:15.737989 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:15.737989 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:49:15.739089 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:49:15.739089 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:49:15.739089 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:15.740117 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 484/972] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:49:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:49:15.745778 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3840a4-1f71-433d-b413-a91fb731616d None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:15.814657 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:15.857550 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3840a4-1f71-433d-b413-a91fb731616d None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:15.887583 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a8589-27f6-4a49-a259-808cfeb14881 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:15.896287 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6849700c-845a-4ab7-80b0-bae0a8e11a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:15.896942 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849700c-845a-4ab7-80b0-bae0a8e11a2e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:15.897226 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849700c-845a-4ab7-80b0-bae0a8e11a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:15.897505 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849700c-845a-4ab7-80b0-bae0a8e11a2e None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:15.898157 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6849700c-845a-4ab7-80b0-bae0a8e11a2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:15.898463 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6849700c-845a-4ab7-80b0-bae0a8e11a2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:15.900052 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6849700c-845a-4ab7-80b0-bae0a8e11a2e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:15.912384 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 489/973] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:49:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:15.918243 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6849700c-845a-4ab7-80b0-bae0a8e11a2e None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:15.999364 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a8589-27f6-4a49-a259-808cfeb14881 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:16.030534 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:16.036907 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461cf4d-d142-48a2-b227-727a73041bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:16.037534 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461cf4d-d142-48a2-b227-727a73041bca None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:16.037809 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461cf4d-d142-48a2-b227-727a73041bca None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:16.038086 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461cf4d-d142-48a2-b227-727a73041bca None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:16.039069 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f461cf4d-d142-48a2-b227-727a73041bca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:16.040796 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f461cf4d-d142-48a2-b227-727a73041bca None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:16.059281 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6849700c-845a-4ab7-80b0-bae0a8e11a2e None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:16.085289 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:16.088064 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.sql.core [None req-f461cf4d-d142-48a2-b227-727a73041bca None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 04 00:49:16.088064 dv-cinder-master-3-smb devstack@keystone.service[81629]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 04 00:49:16.088064 dv-cinder-master-3-smb devstack@keystone.service[81629]: [parameters: {'user_id': 'ba9ceb194d56425a9c92e0310a3e87ea', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 04 00:49:16.088064 dv-cinder-master-3-smb devstack@keystone.service[81629]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81629) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 04 00:49:16.097416 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d9abfb-9c4f-40df-b355-2c743861a9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:16.099450 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9abfb-9c4f-40df-b355-2c743861a9fc None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:16.099450 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9abfb-9c4f-40df-b355-2c743861a9fc None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:16.099450 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9abfb-9c4f-40df-b355-2c743861a9fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:16.099891 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-f461cf4d-d142-48a2-b227-727a73041bca None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 04 00:49:16.099891 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:16.099891 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 04 00:49:16.099891 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 04 00:49:16.099891 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 04 00:49:16.099891 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 04 00:49:16.099891 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 04 00:49:16.099891 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application result = self._query(query) Sep 04 00:49:16.099891 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 04 00:49:16.099891 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application conn.query(q) Sep 04 00:49:16.099891 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 04 00:49:16.099891 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 04 00:49:16.099891 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 04 00:49:16.099891 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application result.read() Sep 04 00:49:16.099891 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 04 00:49:16.100914 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 04 00:49:16.100914 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 04 00:49:16.100914 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 04 00:49:16.100914 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 04 00:49:16.100914 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 04 00:49:16.100914 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 04 00:49:16.100914 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 04 00:49:16.100914 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 04 00:49:16.100914 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:16.100914 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 04 00:49:16.100914 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:16.100914 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:16.100914 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 04 00:49:16.100914 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 04 00:49:16.100914 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 04 00:49:16.100914 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 04 00:49:16.100914 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 04 00:49:16.100914 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application next(self.gen) Sep 04 00:49:16.101882 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 04 00:49:16.101882 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application yield resource Sep 04 00:49:16.101882 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 04 00:49:16.101882 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application next(self.gen) Sep 04 00:49:16.101882 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 04 00:49:16.101882 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self.session.rollback() Sep 04 00:49:16.101882 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 04 00:49:16.101882 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self.force_reraise() Sep 04 00:49:16.101882 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 04 00:49:16.101882 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise self.value Sep 04 00:49:16.101882 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 04 00:49:16.101882 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 04 00:49:16.101882 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 04 00:49:16.101882 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application session.commit() Sep 04 00:49:16.101882 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 04 00:49:16.101882 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 04 00:49:16.101882 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 04 00:49:16.102794 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._prepare_impl() Sep 04 00:49:16.102794 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 04 00:49:16.102794 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self.session.flush() Sep 04 00:49:16.102794 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 04 00:49:16.102794 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._flush(objects) Sep 04 00:49:16.102794 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 04 00:49:16.102794 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 04 00:49:16.102794 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 04 00:49:16.102794 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application compat.raise_( Sep 04 00:49:16.102794 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 04 00:49:16.102794 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception Sep 04 00:49:16.102794 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 04 00:49:16.102794 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application flush_context.execute() Sep 04 00:49:16.102794 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 04 00:49:16.102794 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rec.execute(self) Sep 04 00:49:16.102794 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 04 00:49:16.102794 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 04 00:49:16.103688 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 04 00:49:16.103688 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 04 00:49:16.103688 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 04 00:49:16.103688 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 04 00:49:16.103688 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 04 00:49:16.103688 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 04 00:49:16.103688 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 04 00:49:16.103688 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 04 00:49:16.103688 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 04 00:49:16.103688 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 04 00:49:16.103688 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 04 00:49:16.103688 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 04 00:49:16.103688 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 04 00:49:16.103688 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 04 00:49:16.103688 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 04 00:49:16.103688 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception Sep 04 00:49:16.104738 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 04 00:49:16.104738 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 04 00:49:16.104738 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 04 00:49:16.104738 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 04 00:49:16.104738 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 04 00:49:16.104738 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application result = self._query(query) Sep 04 00:49:16.104738 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 04 00:49:16.104738 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application conn.query(q) Sep 04 00:49:16.104738 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 04 00:49:16.104738 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 04 00:49:16.104738 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 04 00:49:16.104738 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application result.read() Sep 04 00:49:16.104738 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 04 00:49:16.104738 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 04 00:49:16.104738 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 04 00:49:16.104738 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 04 00:49:16.110183 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 04 00:49:16.110183 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 04 00:49:16.110183 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 04 00:49:16.110183 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 04 00:49:16.110183 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 04 00:49:16.110183 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 04 00:49:16.110183 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ba9ceb194d56425a9c92e0310a3e87ea', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 04 00:49:16.110183 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 04 00:49:16.110183 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:16.110183 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 00:49:16.110183 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:16.110183 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:16.110183 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:16.110183 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:16.110183 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:16.111988 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:16.111988 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:16.111988 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:16.111988 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:16.111988 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:16.111988 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:16.111988 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:16.111988 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 04 00:49:16.111988 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 04 00:49:16.111988 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:16.111988 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:16.111988 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:49:16.111988 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:49:16.111988 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:49:16.111988 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:49:16.111988 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 04 00:49:16.111988 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 04 00:49:16.114041 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 04 00:49:16.114041 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 04 00:49:16.114041 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 04 00:49:16.114041 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 04 00:49:16.114041 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 04 00:49:16.114041 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:16.114041 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 485/974] 10.222.0.27 () {60 vars in 1131 bytes} [Sun Sep 4 00:49:15 2022] POST /identity/v3/users => generated 160 bytes in 214 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:16.114041 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461cf4d-d142-48a2-b227-727a73041bca None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:16.114984 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d9abfb-9c4f-40df-b355-2c743861a9fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:16.124515 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d9abfb-9c4f-40df-b355-2c743861a9fc None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:16.129544 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-73d9abfb-9c4f-40df-b355-2c743861a9fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:49:16.129544 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:16.129544 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:16.129544 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:16.129544 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:16.129544 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:16.129544 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:16.129544 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:16.129544 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:16.129544 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:16.129544 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:16.129544 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:16.129544 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:49:16.129544 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:49:16.129544 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:49:16.131345 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:49:16.131345 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:16.131345 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:16.131345 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:16.131345 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:16.131345 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:16.131345 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:16.131345 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:16.131345 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:16.131345 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:16.131345 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:16.131345 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:16.131345 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:16.131345 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:16.131345 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:16.131345 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:16.131345 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:16.132971 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:49:16.132971 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:49:16.132971 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:49:16.132971 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:49:16.132971 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:49:16.132971 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:49:16.132971 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:49:16.132971 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:16.136659 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 490/975] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:49:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:16.137330 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d9abfb-9c4f-40df-b355-2c743861a9fc None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:16.240165 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461cf4d-d142-48a2-b227-727a73041bca None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:16.259120 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d9abfb-9c4f-40df-b355-2c743861a9fc None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:16.286986 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec1bf7-c2ad-49d9-b411-67a9bdc8b194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:16.288127 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec1bf7-c2ad-49d9-b411-67a9bdc8b194 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:16.288814 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec1bf7-c2ad-49d9-b411-67a9bdc8b194 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:16.289089 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec1bf7-c2ad-49d9-b411-67a9bdc8b194 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:16.296225 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ec9fd-6bbd-4aeb-8dd3-f419f24e1d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:16.297641 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ec9fd-6bbd-4aeb-8dd3-f419f24e1d08 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:16.298033 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ec9fd-6bbd-4aeb-8dd3-f419f24e1d08 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:16.298452 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ec9fd-6bbd-4aeb-8dd3-f419f24e1d08 None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:16.298579 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cec1bf7-c2ad-49d9-b411-67a9bdc8b194 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:16.299329 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ec9fd-6bbd-4aeb-8dd3-f419f24e1d08 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:16.299733 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ec9fd-6bbd-4aeb-8dd3-f419f24e1d08 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:16.300160 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cec1bf7-c2ad-49d9-b411-67a9bdc8b194 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:16.301571 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ec9fd-6bbd-4aeb-8dd3-f419f24e1d08 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:16.315867 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-2cec1bf7-c2ad-49d9-b411-67a9bdc8b194 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 04 00:49:16.315867 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:16.315867 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:16.315867 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:16.315867 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:16.315867 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:16.315867 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:16.315867 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:16.315867 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:16.315867 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:16.315867 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:16.315867 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:16.315867 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:49:16.315867 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:49:16.315867 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:49:16.315867 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:49:16.317098 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:16.317098 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:16.317098 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:49:16.317098 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:49:16.317098 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:49:16.317098 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:49:16.317098 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:16.317098 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:16.317098 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:16.317098 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:16.317098 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:16.317098 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:16.317098 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:16.317098 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:16.317098 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:16.317098 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:16.317098 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:16.317098 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:16.318424 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:16.318424 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:16.318424 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:49:16.318424 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:49:16.318424 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:49:16.318424 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:49:16.318424 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:49:16.318424 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:49:16.318424 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 04 00:49:16.318424 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:16.318424 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 486/976] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:49:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:16.324487 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 491/977] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:49:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:16.324645 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec1bf7-c2ad-49d9-b411-67a9bdc8b194 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:16.329344 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ec9fd-6bbd-4aeb-8dd3-f419f24e1d08 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:16.382393 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:16.397271 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:16.448070 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ec9fd-6bbd-4aeb-8dd3-f419f24e1d08 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:16.459939 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec1bf7-c2ad-49d9-b411-67a9bdc8b194 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:16.487461 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd004691-28d0-4130-bc46-0f0b6a53594a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:16.488565 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd004691-28d0-4130-bc46-0f0b6a53594a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:16.489517 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd004691-28d0-4130-bc46-0f0b6a53594a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:16.490651 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd004691-28d0-4130-bc46-0f0b6a53594a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:16.491598 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd004691-28d0-4130-bc46-0f0b6a53594a None admin] RBAC: Adding query filter params (role.id=91d340dbd1f142c5aa33ad640eca891c, scope.project.id=5100a8d316d243fb8f82e9044e1b3b06, user.id=d6d01ca673a1404c9b4cea51036c595c) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:16.492108 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd004691-28d0-4130-bc46-0f0b6a53594a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:16.493996 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd004691-28d0-4130-bc46-0f0b6a53594a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:16.508828 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 492/978] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:49:16 2022] GET /identity/v3/role_assignments?user.id=d6d01ca673a1404c9b4cea51036c595c&scope.project.id=5100a8d316d243fb8f82e9044e1b3b06&role.id=91d340dbd1f142c5aa33ad640eca891c => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:16.512085 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2300601-34dc-4332-a988-bf8efd51106c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:16.512809 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2300601-34dc-4332-a988-bf8efd51106c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:16.513093 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2300601-34dc-4332-a988-bf8efd51106c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:16.513365 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2300601-34dc-4332-a988-bf8efd51106c None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:16.514081 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2300601-34dc-4332-a988-bf8efd51106c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:16.514384 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2300601-34dc-4332-a988-bf8efd51106c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:16.516518 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2300601-34dc-4332-a988-bf8efd51106c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:16.564096 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 487/979] 10.222.0.27 () {56 vars in 1101 bytes} [Sun Sep 4 00:49:16 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:16.945342 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125d0f2-9fa8-4c0f-a5ad-0edd7a74a1b7 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:16.945674 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125d0f2-9fa8-4c0f-a5ad-0edd7a74a1b7 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:16.945906 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125d0f2-9fa8-4c0f-a5ad-0edd7a74a1b7 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:16.947385 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 493/980] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:49:16.952477 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db00c9c-9873-4377-bddb-790cf9df9a79 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:16.953406 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db00c9c-9873-4377-bddb-790cf9df9a79 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:16.953852 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db00c9c-9873-4377-bddb-790cf9df9a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:16.972644 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-4db00c9c-9873-4377-bddb-790cf9df9a79 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:16.974337 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4db00c9c-9873-4377-bddb-790cf9df9a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:17.044028 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 488/981] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:49:17.316388 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7125d0f2-9fa8-4c0f-a5ad-0edd7a74a1b7 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:17.379314 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3da1a-175b-4db2-b8dd-2909c288394f None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:17.379652 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3da1a-175b-4db2-b8dd-2909c288394f None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:17.379875 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3da1a-175b-4db2-b8dd-2909c288394f None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:17.381541 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 489/982] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 00:49:17.386152 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50606756-643c-4bc0-9f39-de4c3cb378d8 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:17.386440 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50606756-643c-4bc0-9f39-de4c3cb378d8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:17.386919 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50606756-643c-4bc0-9f39-de4c3cb378d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:17.405952 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-50606756-643c-4bc0-9f39-de4c3cb378d8 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:17.407204 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-50606756-643c-4bc0-9f39-de4c3cb378d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:17.431267 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7125d0f2-9fa8-4c0f-a5ad-0edd7a74a1b7 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:17.441475 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:17.444046 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:17.465947 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c662596a-295c-4411-9159-6d69a8ebfddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:17.466547 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662596a-295c-4411-9159-6d69a8ebfddc None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:17.466823 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662596a-295c-4411-9159-6d69a8ebfddc None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:17.467098 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662596a-295c-4411-9159-6d69a8ebfddc None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:17.467808 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c662596a-295c-4411-9159-6d69a8ebfddc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:17.469276 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c662596a-295c-4411-9159-6d69a8ebfddc None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:17.477959 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 490/983] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 00:49:17.546059 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 494/984] 10.222.0.27 () {60 vars in 1141 bytes} [Sun Sep 4 00:49:17 2022] POST /identity/v3/services => generated 242 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:49:17.744095 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50606756-643c-4bc0-9f39-de4c3cb378d8 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:17.846135 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50606756-643c-4bc0-9f39-de4c3cb378d8 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:17.886965 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe8e44-7955-4486-ab29-d6b0db9413df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:17.894944 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe8e44-7955-4486-ab29-d6b0db9413df None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:17.894944 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe8e44-7955-4486-ab29-d6b0db9413df None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:17.894944 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe8e44-7955-4486-ab29-d6b0db9413df None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:17.894944 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abe8e44-7955-4486-ab29-d6b0db9413df None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:17.894944 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abe8e44-7955-4486-ab29-d6b0db9413df None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:17.899142 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-9abe8e44-7955-4486-ab29-d6b0db9413df None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 04 00:49:17.899142 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:17.899142 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 00:49:17.899142 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 00:49:17.899142 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 00:49:17.899142 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 00:49:17.899142 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 00:49:17.899142 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:49:17.899142 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 04 00:49:17.899142 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:17.899142 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 00:49:17.899142 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:17.899142 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:17.899142 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:17.899142 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:17.899142 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:17.899142 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:17.900292 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:17.900292 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:17.900292 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:17.900292 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:17.900292 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:17.900292 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:17.900292 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 00:49:17.900292 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 00:49:17.900292 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 00:49:17.900292 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 00:49:17.900292 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:17.900292 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:17.900292 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:17.900292 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:17.900292 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:17.900292 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:17.901520 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:17.901520 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:17.901520 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:17.901520 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:17.901520 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:17.901520 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:17.901520 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:17.901520 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:17.901520 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:17.901520 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:17.901520 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 00:49:17.901520 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:49:17.901520 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 04 00:49:17.901520 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:17.902643 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 491/985] 10.222.0.27 () {56 vars in 1126 bytes} [Sun Sep 4 00:49:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:17.906659 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c662596a-295c-4411-9159-6d69a8ebfddc None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:18.008996 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c662596a-295c-4411-9159-6d69a8ebfddc None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:18.042735 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c192a9-ac98-4d69-b670-c05f182358fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:18.043345 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c192a9-ac98-4d69-b670-c05f182358fc None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:18.043622 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c192a9-ac98-4d69-b670-c05f182358fc None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:18.043888 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c192a9-ac98-4d69-b670-c05f182358fc None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:18.044535 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c192a9-ac98-4d69-b670-c05f182358fc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:18.044841 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c192a9-ac98-4d69-b670-c05f182358fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:18.046262 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c192a9-ac98-4d69-b670-c05f182358fc None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:18.056559 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 495/986] 10.222.0.27 () {56 vars in 1122 bytes} [Sun Sep 4 00:49:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:18.057520 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe8e44-7955-4486-ab29-d6b0db9413df None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:18.090017 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:18.160592 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe8e44-7955-4486-ab29-d6b0db9413df None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:18.194820 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3d72d-4def-4915-ba87-55bb3e3415a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:18.195423 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3d72d-4def-4915-ba87-55bb3e3415a0 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:18.195701 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3d72d-4def-4915-ba87-55bb3e3415a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:18.195971 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3d72d-4def-4915-ba87-55bb3e3415a0 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:18.196620 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c3d72d-4def-4915-ba87-55bb3e3415a0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:18.196920 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c3d72d-4def-4915-ba87-55bb3e3415a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:18.198260 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c3d72d-4def-4915-ba87-55bb3e3415a0 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:18.203423 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 492/987] 10.222.0.27 () {56 vars in 1122 bytes} [Sun Sep 4 00:49:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:49:18.209070 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c192a9-ac98-4d69-b670-c05f182358fc None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:18.248750 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:18.317673 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c192a9-ac98-4d69-b670-c05f182358fc None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:18.351206 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064a2c1-b749-4627-8b85-a12626b63626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:18.351812 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064a2c1-b749-4627-8b85-a12626b63626 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:18.352095 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064a2c1-b749-4627-8b85-a12626b63626 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:18.352364 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064a2c1-b749-4627-8b85-a12626b63626 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:18.353092 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b064a2c1-b749-4627-8b85-a12626b63626 None admin] RBAC: Adding query filter params (interface=admin, service_id=b3d567d3ce664858b753ae51f5d61605, region_id=RegionOne) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:18.353399 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b064a2c1-b749-4627-8b85-a12626b63626 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:18.354756 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b064a2c1-b749-4627-8b85-a12626b63626 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:18.365101 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 496/988] 10.222.0.27 () {56 vars in 1249 bytes} [Sun Sep 4 00:49:18 2022] GET /identity/v3/endpoints?service_id=b3d567d3ce664858b753ae51f5d61605&interface=admin®ion_id=RegionOne => generated 193 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:18.366201 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3d72d-4def-4915-ba87-55bb3e3415a0 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:18.466921 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:18.473980 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3d72d-4def-4915-ba87-55bb3e3415a0 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:18.507614 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e199ad-5e5a-4ad4-bd9e-67d0de134faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:18.508225 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e199ad-5e5a-4ad4-bd9e-67d0de134faa None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:18.508531 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e199ad-5e5a-4ad4-bd9e-67d0de134faa None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:18.508718 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413bf8a-4666-4cea-8656-85e555ca7579 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:18.508831 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e199ad-5e5a-4ad4-bd9e-67d0de134faa None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:18.509007 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413bf8a-4666-4cea-8656-85e555ca7579 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:18.509270 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413bf8a-4666-4cea-8656-85e555ca7579 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:18.509384 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e199ad-5e5a-4ad4-bd9e-67d0de134faa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:18.510798 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 497/989] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:49:18.510903 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e199ad-5e5a-4ad4-bd9e-67d0de134faa None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:18.515507 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3d703-3286-40ea-8941-d06c702d8b45 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:18.515986 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3d703-3286-40ea-8941-d06c702d8b45 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:18.516437 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3d703-3286-40ea-8941-d06c702d8b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:18.524867 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 493/990] 10.222.0.27 () {56 vars in 1087 bytes} [Sun Sep 4 00:49:18 2022] GET /identity/v3/services => generated 1727 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 00:49:18.545377 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-b9f3d703-3286-40ea-8941-d06c702d8b45 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:18.547135 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b9f3d703-3286-40ea-8941-d06c702d8b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:18.620502 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 498/991] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:18 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 00:49:18.896384 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e199ad-5e5a-4ad4-bd9e-67d0de134faa None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:18.999816 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e199ad-5e5a-4ad4-bd9e-67d0de134faa None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:19.033828 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee4a26-6519-4c0e-a168-4beb60c54544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:19.034506 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee4a26-6519-4c0e-a168-4beb60c54544 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:19.034782 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee4a26-6519-4c0e-a168-4beb60c54544 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:19.035052 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee4a26-6519-4c0e-a168-4beb60c54544 None admin] PATH_INFO: `/v3/services/image` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:19.035676 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ee4a26-6519-4c0e-a168-4beb60c54544 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:19.037057 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ee4a26-6519-4c0e-a168-4beb60c54544 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:19.046017 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-d7ee4a26-6519-4c0e-a168-4beb60c54544 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 04 00:49:19.046017 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:19.046017 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 00:49:19.046017 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 00:49:19.046017 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 00:49:19.046017 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 00:49:19.046017 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 00:49:19.046017 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:49:19.046017 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 04 00:49:19.046017 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:19.046017 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 00:49:19.046017 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:19.046017 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:19.046017 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:19.046017 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:19.046017 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:19.046017 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:19.047771 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:19.047771 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:19.047771 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:19.047771 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:19.047771 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:19.047771 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:19.047771 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 00:49:19.047771 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 00:49:19.047771 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 00:49:19.047771 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 00:49:19.047771 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:19.047771 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:19.047771 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:19.047771 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:19.047771 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:19.047771 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:19.049417 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:19.049417 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:19.049417 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:19.049417 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:19.049417 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:19.049417 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:19.049417 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:19.049417 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:19.049417 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:19.049417 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:19.049417 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 00:49:19.049417 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:49:19.049417 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 04 00:49:19.049417 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:19.055784 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 494/992] 10.222.0.27 () {56 vars in 1105 bytes} [Sun Sep 4 00:49:18 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:19.055933 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f3d703-3286-40ea-8941-d06c702d8b45 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:19.082050 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:19.156553 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f3d703-3286-40ea-8941-d06c702d8b45 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:19.191595 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e293b89d-c7df-4aa9-bf49-496dbb5cd1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:19.192277 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293b89d-c7df-4aa9-bf49-496dbb5cd1ba None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:19.192587 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293b89d-c7df-4aa9-bf49-496dbb5cd1ba None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:19.192862 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293b89d-c7df-4aa9-bf49-496dbb5cd1ba None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:19.193483 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e293b89d-c7df-4aa9-bf49-496dbb5cd1ba None admin] RBAC: Adding query filter params (name=image) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:19.193783 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e293b89d-c7df-4aa9-bf49-496dbb5cd1ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:19.195234 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e293b89d-c7df-4aa9-bf49-496dbb5cd1ba None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:19.204617 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 499/993] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:49:19 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:49:19.206439 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee4a26-6519-4c0e-a168-4beb60c54544 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:19.241274 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:19.317518 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee4a26-6519-4c0e-a168-4beb60c54544 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:19.352116 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757f9ab-0fc7-4476-beed-763c0412d079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:19.352745 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757f9ab-0fc7-4476-beed-763c0412d079 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:19.353024 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757f9ab-0fc7-4476-beed-763c0412d079 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:19.353295 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757f9ab-0fc7-4476-beed-763c0412d079 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:19.353914 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f757f9ab-0fc7-4476-beed-763c0412d079 None admin] RBAC: Adding query filter params (type=image) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:19.354214 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f757f9ab-0fc7-4476-beed-763c0412d079 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:19.355556 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f757f9ab-0fc7-4476-beed-763c0412d079 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:19.364670 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 495/994] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:49:19 2022] GET /identity/v3/services?type=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:19.492627 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34cd9cd-47a6-4c5d-8f4a-987f9edf0c3d None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:19.493107 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34cd9cd-47a6-4c5d-8f4a-987f9edf0c3d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:19.493107 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34cd9cd-47a6-4c5d-8f4a-987f9edf0c3d None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:19.494537 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 500/995] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:49:19.499619 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bca687-3d93-4009-9e0b-aa1ea0324032 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:19.499962 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bca687-3d93-4009-9e0b-aa1ea0324032 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:19.500341 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bca687-3d93-4009-9e0b-aa1ea0324032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:19.520951 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-04bca687-3d93-4009-9e0b-aa1ea0324032 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:19.522160 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-04bca687-3d93-4009-9e0b-aa1ea0324032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:19.591762 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 496/996] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:19 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:49:19.861753 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34cd9cd-47a6-4c5d-8f4a-987f9edf0c3d None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:19.953432 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:19.963553 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34cd9cd-47a6-4c5d-8f4a-987f9edf0c3d None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:19.997803 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7109905e-915c-4bdc-9679-eba5ffbe2398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:19.998434 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109905e-915c-4bdc-9679-eba5ffbe2398 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:19.998709 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109905e-915c-4bdc-9679-eba5ffbe2398 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:19.998976 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109905e-915c-4bdc-9679-eba5ffbe2398 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:20.000156 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7109905e-915c-4bdc-9679-eba5ffbe2398 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:20.001552 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7109905e-915c-4bdc-9679-eba5ffbe2398 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:20.010356 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-7109905e-915c-4bdc-9679-eba5ffbe2398 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 04 00:49:20.010356 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:20.010356 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 00:49:20.010356 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 00:49:20.010356 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 00:49:20.010356 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 00:49:20.010356 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 00:49:20.010356 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:49:20.010356 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 04 00:49:20.010356 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:20.010356 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 00:49:20.010356 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:20.010356 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:20.010356 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:20.010356 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:20.010356 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:20.010356 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:20.011388 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:20.011388 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:20.011388 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:20.011388 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:20.011388 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:20.011388 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:20.011388 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 00:49:20.011388 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 00:49:20.011388 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 00:49:20.011388 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 00:49:20.011388 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:20.011388 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:20.011388 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:20.011388 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:20.011388 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:20.011388 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:20.012247 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:20.012247 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:20.012247 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:20.012247 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:20.012247 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:20.012247 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:20.012247 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:20.012247 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:20.012247 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:20.012247 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:20.012247 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 00:49:20.012247 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:49:20.012247 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 04 00:49:20.012247 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:20.019404 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 501/997] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:49:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:20.019550 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bca687-3d93-4009-9e0b-aa1ea0324032 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:20.035136 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:20.119207 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bca687-3d93-4009-9e0b-aa1ea0324032 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:20.153522 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a8502-2458-4f39-bac9-3741e261eb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:20.154121 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a8502-2458-4f39-bac9-3741e261eb23 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:20.154402 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a8502-2458-4f39-bac9-3741e261eb23 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:20.154667 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a8502-2458-4f39-bac9-3741e261eb23 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:20.155277 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756a8502-2458-4f39-bac9-3741e261eb23 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:20.155575 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756a8502-2458-4f39-bac9-3741e261eb23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:20.156960 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756a8502-2458-4f39-bac9-3741e261eb23 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:20.161798 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 497/998] 10.222.0.27 () {56 vars in 1112 bytes} [Sun Sep 4 00:49:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:20.167465 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7109905e-915c-4bdc-9679-eba5ffbe2398 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:20.278039 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7109905e-915c-4bdc-9679-eba5ffbe2398 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:20.313001 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec7642-8499-4561-bc5c-8cc23f36140a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:20.313693 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec7642-8499-4561-bc5c-8cc23f36140a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:20.313953 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec7642-8499-4561-bc5c-8cc23f36140a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:20.314146 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec7642-8499-4561-bc5c-8cc23f36140a None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:20.314790 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bec7642-8499-4561-bc5c-8cc23f36140a None admin] RBAC: Adding query filter params (type=compute) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:20.315098 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bec7642-8499-4561-bc5c-8cc23f36140a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:20.316496 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bec7642-8499-4561-bc5c-8cc23f36140a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:20.326361 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 502/999] 10.222.0.27 () {56 vars in 1112 bytes} [Sun Sep 4 00:49:20 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:49:20.327324 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a8502-2458-4f39-bac9-3741e261eb23 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:20.412219 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:20.432868 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a8502-2458-4f39-bac9-3741e261eb23 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:20.460984 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de968f2-a959-4523-9fbc-a848515e60cb None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:20.461336 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de968f2-a959-4523-9fbc-a848515e60cb None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:20.461559 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de968f2-a959-4523-9fbc-a848515e60cb None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:20.463017 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 503/1000] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 00:49:20.467697 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473576cd-46a3-40f3-9d37-e3700428d95a None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:20.467997 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473576cd-46a3-40f3-9d37-e3700428d95a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:20.468825 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473576cd-46a3-40f3-9d37-e3700428d95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:20.469387 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c31d10-3725-4866-81de-79913499e632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:20.470017 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c31d10-3725-4866-81de-79913499e632 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:20.470297 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c31d10-3725-4866-81de-79913499e632 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:20.470566 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c31d10-3725-4866-81de-79913499e632 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:20.471252 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c31d10-3725-4866-81de-79913499e632 None admin] RBAC: Adding query filter params (interface=public, service_id=3dbaecfce8b84c7795eb0279df475271, region_id=RegionOne) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:20.471557 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c31d10-3725-4866-81de-79913499e632 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:20.472918 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c31d10-3725-4866-81de-79913499e632 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:20.479290 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 498/1001] 10.222.0.27 () {56 vars in 1251 bytes} [Sun Sep 4 00:49:20 2022] GET /identity/v3/endpoints?service_id=3dbaecfce8b84c7795eb0279df475271&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:49:20.485159 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c31d10-3725-4866-81de-79913499e632 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:20.488064 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-473576cd-46a3-40f3-9d37-e3700428d95a None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:20.489455 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-473576cd-46a3-40f3-9d37-e3700428d95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:20.503096 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:20.559179 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 504/1002] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:20 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 00:49:20.595599 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c31d10-3725-4866-81de-79913499e632 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:20.629536 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916791bb-a689-4cae-ab29-34273c6faed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:20.630148 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916791bb-a689-4cae-ab29-34273c6faed5 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:20.630429 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916791bb-a689-4cae-ab29-34273c6faed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:20.630698 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916791bb-a689-4cae-ab29-34273c6faed5 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:20.631282 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916791bb-a689-4cae-ab29-34273c6faed5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:20.632653 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916791bb-a689-4cae-ab29-34273c6faed5 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:20.638993 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 499/1003] 10.222.0.27 () {56 vars in 1087 bytes} [Sun Sep 4 00:49:20 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 00:49:20.833021 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473576cd-46a3-40f3-9d37-e3700428d95a None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:20.938360 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473576cd-46a3-40f3-9d37-e3700428d95a None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:20.973801 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72deb4f6-4401-4a2b-9c87-10748e3ef7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:20.974466 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72deb4f6-4401-4a2b-9c87-10748e3ef7d1 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:20.974750 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72deb4f6-4401-4a2b-9c87-10748e3ef7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:20.975026 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72deb4f6-4401-4a2b-9c87-10748e3ef7d1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:20.975614 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72deb4f6-4401-4a2b-9c87-10748e3ef7d1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:20.977037 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72deb4f6-4401-4a2b-9c87-10748e3ef7d1 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:20.986532 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-72deb4f6-4401-4a2b-9c87-10748e3ef7d1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 04 00:49:20.986532 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:20.986532 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 00:49:20.986532 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 00:49:20.986532 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 00:49:20.986532 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 00:49:20.986532 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 00:49:20.986532 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:49:20.986532 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 04 00:49:20.986532 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:20.986532 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 00:49:20.986532 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:20.986532 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:20.986532 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:20.986532 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:20.986532 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:20.986532 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:20.987534 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:20.987534 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:20.987534 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:20.987534 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:20.987534 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:20.987534 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:20.987534 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 00:49:20.987534 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 00:49:20.987534 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 00:49:20.987534 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 00:49:20.987534 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:20.987534 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:20.987534 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:20.987534 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:20.987534 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:20.987534 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:20.988369 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:20.988369 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:20.988369 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:20.988369 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:20.988369 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:20.988369 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:20.988369 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:20.988369 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:20.988369 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:20.988369 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:20.988369 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 00:49:20.988369 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:49:20.988369 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 04 00:49:20.988369 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:20.992511 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 505/1004] 10.222.0.27 () {56 vars in 1126 bytes} [Sun Sep 4 00:49:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:20.993924 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916791bb-a689-4cae-ab29-34273c6faed5 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:21.005732 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:21.098623 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916791bb-a689-4cae-ab29-34273c6faed5 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:21.133477 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d504724-dce3-456c-ad69-47a6a15de9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:21.134084 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d504724-dce3-456c-ad69-47a6a15de9ad None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:21.134369 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d504724-dce3-456c-ad69-47a6a15de9ad None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:21.134645 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d504724-dce3-456c-ad69-47a6a15de9ad None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:21.135270 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d504724-dce3-456c-ad69-47a6a15de9ad None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:21.135575 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d504724-dce3-456c-ad69-47a6a15de9ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:21.136980 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d504724-dce3-456c-ad69-47a6a15de9ad None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:21.146479 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 500/1005] 10.222.0.27 () {56 vars in 1122 bytes} [Sun Sep 4 00:49:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:21.147695 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72deb4f6-4401-4a2b-9c87-10748e3ef7d1 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:21.165176 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:21.259934 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72deb4f6-4401-4a2b-9c87-10748e3ef7d1 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:21.274490 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8e604-8eb9-456a-b8ab-3e231100b14c None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:21.274769 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8e604-8eb9-456a-b8ab-3e231100b14c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:21.275062 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8e604-8eb9-456a-b8ab-3e231100b14c None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:21.276553 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 501/1006] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 00:49:21.281662 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6587f-8489-4d2a-b5ca-0de7b7dee1a6 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:21.281949 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6587f-8489-4d2a-b5ca-0de7b7dee1a6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:21.282229 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6587f-8489-4d2a-b5ca-0de7b7dee1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:21.298082 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde4420-110f-4d1e-9fe8-27e1f7d556d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:21.298696 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde4420-110f-4d1e-9fe8-27e1f7d556d7 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:21.298981 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde4420-110f-4d1e-9fe8-27e1f7d556d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:21.299258 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde4420-110f-4d1e-9fe8-27e1f7d556d7 None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:21.299886 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde4420-110f-4d1e-9fe8-27e1f7d556d7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:21.300193 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde4420-110f-4d1e-9fe8-27e1f7d556d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:21.301599 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde4420-110f-4d1e-9fe8-27e1f7d556d7 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:21.301765 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-dea6587f-8489-4d2a-b5ca-0de7b7dee1a6 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:21.302916 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-dea6587f-8489-4d2a-b5ca-0de7b7dee1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:21.311182 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 506/1007] 10.222.0.27 () {56 vars in 1122 bytes} [Sun Sep 4 00:49:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:21.312248 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde4420-110f-4d1e-9fe8-27e1f7d556d7 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:21.375745 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 502/1008] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:21 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 00:49:21.389258 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:21.412901 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde4420-110f-4d1e-9fe8-27e1f7d556d7 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:21.446897 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f2b281-de01-4bbe-8003-d83b5c49806d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:21.447500 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f2b281-de01-4bbe-8003-d83b5c49806d None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:21.447784 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f2b281-de01-4bbe-8003-d83b5c49806d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:21.448059 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f2b281-de01-4bbe-8003-d83b5c49806d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:21.448743 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f2b281-de01-4bbe-8003-d83b5c49806d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:21.450124 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f2b281-de01-4bbe-8003-d83b5c49806d None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:21.548495 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 507/1009] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 00:49:21 2022] POST /identity/v3/endpoints => generated 325 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:49:21.643453 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6587f-8489-4d2a-b5ca-0de7b7dee1a6 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:21.645622 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:21.754787 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6587f-8489-4d2a-b5ca-0de7b7dee1a6 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:21.797700 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac94747-613d-47ce-a8eb-fa2e898b330f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:21.798317 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac94747-613d-47ce-a8eb-fa2e898b330f None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:21.798591 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac94747-613d-47ce-a8eb-fa2e898b330f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:21.798860 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac94747-613d-47ce-a8eb-fa2e898b330f None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:21.799508 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac94747-613d-47ce-a8eb-fa2e898b330f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:21.802688 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac94747-613d-47ce-a8eb-fa2e898b330f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:21.920571 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 503/1010] 10.222.0.27 () {60 vars in 1141 bytes} [Sun Sep 4 00:49:21 2022] POST /identity/v3/services => generated 242 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:49:22.555054 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fbf8dd-c79f-4184-a502-ff7346d536a2 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:22.555563 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fbf8dd-c79f-4184-a502-ff7346d536a2 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:22.555664 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fbf8dd-c79f-4184-a502-ff7346d536a2 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:22.556989 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 508/1011] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:49:22.562077 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4e20c-3f73-4110-9f03-66cbc01b4f55 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:22.562458 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4e20c-3f73-4110-9f03-66cbc01b4f55 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:22.562726 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4e20c-3f73-4110-9f03-66cbc01b4f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:22.581014 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-9ee4e20c-3f73-4110-9f03-66cbc01b4f55 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:22.582207 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9ee4e20c-3f73-4110-9f03-66cbc01b4f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:22.637280 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:22.658001 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 504/1012] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:22 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:49:22.924896 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fbf8dd-c79f-4184-a502-ff7346d536a2 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:23.031390 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fbf8dd-c79f-4184-a502-ff7346d536a2 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:23.067815 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b85bf23-6351-46b7-84e2-2ea2f58fb7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:23.068485 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b85bf23-6351-46b7-84e2-2ea2f58fb7e3 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:23.068768 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b85bf23-6351-46b7-84e2-2ea2f58fb7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:23.069053 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b85bf23-6351-46b7-84e2-2ea2f58fb7e3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:23.069645 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b85bf23-6351-46b7-84e2-2ea2f58fb7e3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:23.071029 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b85bf23-6351-46b7-84e2-2ea2f58fb7e3 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:23.073909 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:23.081328 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-2b85bf23-6351-46b7-84e2-2ea2f58fb7e3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 04 00:49:23.081328 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:23.081328 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 00:49:23.081328 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 00:49:23.081328 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 00:49:23.081328 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 00:49:23.081328 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 00:49:23.081328 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:49:23.081328 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 04 00:49:23.081328 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:23.081328 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 00:49:23.081328 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:23.081328 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:23.081328 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:23.081328 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:23.081328 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:23.081328 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:23.082347 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:23.082347 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:23.082347 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:23.082347 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:23.082347 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:23.082347 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:23.082347 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 00:49:23.082347 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 00:49:23.082347 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 00:49:23.082347 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 00:49:23.082347 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:23.082347 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:23.082347 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:23.082347 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:23.082347 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:23.082347 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:23.083190 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:23.083190 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:23.083190 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:23.083190 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:23.083190 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:23.083190 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:23.083190 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:23.083190 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:23.083190 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:23.083190 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:23.083190 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 00:49:23.083190 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:49:23.083190 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 04 00:49:23.083190 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:23.083190 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 509/1013] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:49:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:23.088499 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee4e20c-3f73-4110-9f03-66cbc01b4f55 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:23.189679 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee4e20c-3f73-4110-9f03-66cbc01b4f55 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:23.225754 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7ddde-0402-4610-a9e3-0a18ea692292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:23.226712 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7ddde-0402-4610-a9e3-0a18ea692292 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:23.227200 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7ddde-0402-4610-a9e3-0a18ea692292 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:23.227744 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7ddde-0402-4610-a9e3-0a18ea692292 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:23.228630 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7ddde-0402-4610-a9e3-0a18ea692292 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:23.229154 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7ddde-0402-4610-a9e3-0a18ea692292 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:23.230762 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7ddde-0402-4610-a9e3-0a18ea692292 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:23.241022 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 505/1014] 10.222.0.27 () {56 vars in 1112 bytes} [Sun Sep 4 00:49:23 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:23.242034 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b85bf23-6351-46b7-84e2-2ea2f58fb7e3 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:23.346871 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b85bf23-6351-46b7-84e2-2ea2f58fb7e3 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:23.381638 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50aa220-ab62-4a5c-b3ed-75a2da897fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:23.382244 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50aa220-ab62-4a5c-b3ed-75a2da897fec None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:23.382528 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50aa220-ab62-4a5c-b3ed-75a2da897fec None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:23.382805 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50aa220-ab62-4a5c-b3ed-75a2da897fec None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:23.383436 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50aa220-ab62-4a5c-b3ed-75a2da897fec None admin] RBAC: Adding query filter params (type=compute) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:23.383746 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50aa220-ab62-4a5c-b3ed-75a2da897fec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:23.385145 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50aa220-ab62-4a5c-b3ed-75a2da897fec None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:23.394966 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 510/1015] 10.222.0.27 () {56 vars in 1112 bytes} [Sun Sep 4 00:49:23 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:23.396269 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7ddde-0402-4610-a9e3-0a18ea692292 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:23.447170 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:23.493637 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170cebd7-3c59-444b-9577-c196c2420bc7 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:23.493952 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170cebd7-3c59-444b-9577-c196c2420bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:23.494194 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170cebd7-3c59-444b-9577-c196c2420bc7 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:23.495516 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 511/1016] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 00:49:23.500416 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82609029-6ea5-4b43-9714-265d36896efe None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:23.500796 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82609029-6ea5-4b43-9714-265d36896efe None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:23.501138 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82609029-6ea5-4b43-9714-265d36896efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:23.502565 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7ddde-0402-4610-a9e3-0a18ea692292 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:23.519496 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-82609029-6ea5-4b43-9714-265d36896efe None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:23.520768 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-82609029-6ea5-4b43-9714-265d36896efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:23.538954 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4094656-b2ee-453c-b101-0c5e17bdcce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:23.539556 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4094656-b2ee-453c-b101-0c5e17bdcce1 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:23.539835 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4094656-b2ee-453c-b101-0c5e17bdcce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:23.540126 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4094656-b2ee-453c-b101-0c5e17bdcce1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:23.540798 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4094656-b2ee-453c-b101-0c5e17bdcce1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:23.542133 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4094656-b2ee-453c-b101-0c5e17bdcce1 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:23.582846 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:23.591051 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 512/1017] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:23 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 00:49:23.631338 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 506/1018] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 00:49:23 2022] POST /identity/v3/endpoints => generated 334 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:49:23.871601 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82609029-6ea5-4b43-9714-265d36896efe None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:23.912993 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8825a-9c29-418b-a6aa-c59f0eb2ec55 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:23.913566 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8825a-9c29-418b-a6aa-c59f0eb2ec55 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:23.913820 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8825a-9c29-418b-a6aa-c59f0eb2ec55 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:23.915328 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 507/1019] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:49:23.920072 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cef032-330b-4b92-b3af-535c8d40b5e6 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:23.920390 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cef032-330b-4b92-b3af-535c8d40b5e6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:23.926221 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cef032-330b-4b92-b3af-535c8d40b5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:23.940421 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-75cef032-330b-4b92-b3af-535c8d40b5e6 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:23.941643 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-75cef032-330b-4b92-b3af-535c8d40b5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:23.976650 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82609029-6ea5-4b43-9714-265d36896efe None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:24.018974 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 508/1020] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:23 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 00:49:24.021321 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf4bdc-c570-4b71-9853-b9befb1c5088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:24.021964 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf4bdc-c570-4b71-9853-b9befb1c5088 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:24.022246 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf4bdc-c570-4b71-9853-b9befb1c5088 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:24.022519 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf4bdc-c570-4b71-9853-b9befb1c5088 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:24.023630 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecf4bdc-c570-4b71-9853-b9befb1c5088 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:24.025103 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecf4bdc-c570-4b71-9853-b9befb1c5088 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:24.028493 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 513/1021] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:49:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:24.032514 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cef032-330b-4b92-b3af-535c8d40b5e6 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:24.065629 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:24.136299 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cef032-330b-4b92-b3af-535c8d40b5e6 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:24.173959 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29415a-a57c-4ec5-ac71-d0a46092d89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:24.174575 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29415a-a57c-4ec5-ac71-d0a46092d89a None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:24.174854 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29415a-a57c-4ec5-ac71-d0a46092d89a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:24.175123 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29415a-a57c-4ec5-ac71-d0a46092d89a None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:24.175784 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29415a-a57c-4ec5-ac71-d0a46092d89a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:24.177238 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29415a-a57c-4ec5-ac71-d0a46092d89a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:24.276493 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 509/1022] 10.222.0.27 () {60 vars in 1141 bytes} [Sun Sep 4 00:49:24 2022] POST /identity/v3/projects => generated 311 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:49:24.300583 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf4bdc-c570-4b71-9853-b9befb1c5088 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:24.368442 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:24.414941 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf4bdc-c570-4b71-9853-b9befb1c5088 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:24.452192 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3c732-3bab-4f05-8db3-02e3fd92d0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:24.453414 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3c732-3bab-4f05-8db3-02e3fd92d0e2 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:24.453838 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3c732-3bab-4f05-8db3-02e3fd92d0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:24.454113 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3c732-3bab-4f05-8db3-02e3fd92d0e2 None admin] PATH_INFO: `/v3/services/image` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:24.454694 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3c732-3bab-4f05-8db3-02e3fd92d0e2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:24.456072 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3c732-3bab-4f05-8db3-02e3fd92d0e2 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:24.464999 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-a4d3c732-3bab-4f05-8db3-02e3fd92d0e2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 04 00:49:24.464999 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:24.464999 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 00:49:24.464999 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 00:49:24.464999 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 00:49:24.464999 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 00:49:24.464999 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 00:49:24.464999 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:49:24.464999 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 04 00:49:24.464999 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:24.464999 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 00:49:24.464999 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:24.464999 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:24.464999 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:24.464999 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:24.464999 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:24.464999 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:24.466206 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:24.466206 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:24.466206 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:24.466206 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:24.466206 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:24.466206 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:24.466206 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 00:49:24.466206 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 00:49:24.466206 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 00:49:24.466206 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 00:49:24.466206 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:24.466206 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:24.466206 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:24.466206 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:24.466206 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:24.466206 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:24.467449 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:24.467449 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:24.467449 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:24.467449 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:24.467449 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:24.467449 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:24.467449 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:24.467449 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:24.467449 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:24.467449 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:24.467449 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 00:49:24.467449 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:49:24.467449 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 04 00:49:24.467449 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:24.467449 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 514/1023] 10.222.0.27 () {56 vars in 1105 bytes} [Sun Sep 4 00:49:24 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:24.473030 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29415a-a57c-4ec5-ac71-d0a46092d89a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:24.559676 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:24.578242 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29415a-a57c-4ec5-ac71-d0a46092d89a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:24.612947 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a6c6f-8edc-44a1-95c0-5487679a83fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:24.613559 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a6c6f-8edc-44a1-95c0-5487679a83fe None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:24.613839 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a6c6f-8edc-44a1-95c0-5487679a83fe None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:24.614110 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a6c6f-8edc-44a1-95c0-5487679a83fe None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:24.614741 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a6c6f-8edc-44a1-95c0-5487679a83fe None admin] RBAC: Adding query filter params (name=image) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:24.615045 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a6c6f-8edc-44a1-95c0-5487679a83fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:24.616406 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a6c6f-8edc-44a1-95c0-5487679a83fe None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:24.621672 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 510/1024] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:49:24 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:24.627494 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3c732-3bab-4f05-8db3-02e3fd92d0e2 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:24.733262 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3c732-3bab-4f05-8db3-02e3fd92d0e2 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:24.756607 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:24.770628 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23750afc-14ff-4935-89ae-211076377b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:24.770628 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23750afc-14ff-4935-89ae-211076377b7d None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:24.770628 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23750afc-14ff-4935-89ae-211076377b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:24.770628 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23750afc-14ff-4935-89ae-211076377b7d None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:24.771114 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23750afc-14ff-4935-89ae-211076377b7d None admin] RBAC: Adding query filter params (type=image) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:24.771114 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23750afc-14ff-4935-89ae-211076377b7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:24.772502 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23750afc-14ff-4935-89ae-211076377b7d None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:24.777253 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 515/1025] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:49:24 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:24.783651 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a6c6f-8edc-44a1-95c0-5487679a83fe None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:24.887300 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a6c6f-8edc-44a1-95c0-5487679a83fe None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:24.923186 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f43cf3-587e-4253-8012-5ec7408b2367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:24.923782 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f43cf3-587e-4253-8012-5ec7408b2367 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:24.924058 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f43cf3-587e-4253-8012-5ec7408b2367 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:24.924328 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f43cf3-587e-4253-8012-5ec7408b2367 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:24.925046 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f43cf3-587e-4253-8012-5ec7408b2367 None admin] RBAC: Adding query filter params (interface=public, service_id=edb5690104924d97b8491c92221404b0, region_id=RegionOne) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:24.925347 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f43cf3-587e-4253-8012-5ec7408b2367 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:24.926724 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f43cf3-587e-4253-8012-5ec7408b2367 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:24.932008 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 511/1026] 10.222.0.27 () {56 vars in 1251 bytes} [Sun Sep 4 00:49:24 2022] GET /identity/v3/endpoints?service_id=edb5690104924d97b8491c92221404b0&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:24.937667 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23750afc-14ff-4935-89ae-211076377b7d None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:25.034956 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23750afc-14ff-4935-89ae-211076377b7d None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:25.067720 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5badb0b3-4d06-4b8f-aac2-e547d8ebc6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:25.068318 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5badb0b3-4d06-4b8f-aac2-e547d8ebc6fa None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:25.068645 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5badb0b3-4d06-4b8f-aac2-e547d8ebc6fa None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:25.068918 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5badb0b3-4d06-4b8f-aac2-e547d8ebc6fa None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:25.069512 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5badb0b3-4d06-4b8f-aac2-e547d8ebc6fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:25.070865 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5badb0b3-4d06-4b8f-aac2-e547d8ebc6fa None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:25.078010 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 516/1027] 10.222.0.27 () {56 vars in 1087 bytes} [Sun Sep 4 00:49:24 2022] GET /identity/v3/services => generated 1957 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 00:49:25.656439 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90beb653-e810-4980-9b39-c764af8ce4ec None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:25.656790 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90beb653-e810-4980-9b39-c764af8ce4ec None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:25.657026 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90beb653-e810-4980-9b39-c764af8ce4ec None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:25.658530 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 512/1028] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:49:25.663835 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d511bf-d13c-41be-9004-d7771d54c6bf None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:25.664301 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d511bf-d13c-41be-9004-d7771d54c6bf None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:25.664399 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d511bf-d13c-41be-9004-d7771d54c6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:25.683654 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-d9d511bf-d13c-41be-9004-d7771d54c6bf None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:25.684896 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d9d511bf-d13c-41be-9004-d7771d54c6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:25.733725 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:25.754464 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 517/1029] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:25 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:49:26.024704 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90beb653-e810-4980-9b39-c764af8ce4ec None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:26.037956 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:26.129497 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90beb653-e810-4980-9b39-c764af8ce4ec None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:26.166567 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1caf03-09a4-4f3a-8ea1-e0156043aaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:26.167200 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1caf03-09a4-4f3a-8ea1-e0156043aaa7 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:26.167483 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1caf03-09a4-4f3a-8ea1-e0156043aaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:26.167760 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1caf03-09a4-4f3a-8ea1-e0156043aaa7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:26.178310 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1caf03-09a4-4f3a-8ea1-e0156043aaa7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:49:26.183075 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-ff1caf03-09a4-4f3a-8ea1-e0156043aaa7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 04 00:49:26.183075 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:26.183075 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:26.183075 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:26.183075 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:26.183075 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:26.183075 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:26.183075 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:26.183075 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:26.183075 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:26.183075 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:26.183075 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:26.183075 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:49:26.183075 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:49:26.183075 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:49:26.183075 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:49:26.183958 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:49:26.183958 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:49:26.183958 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:49:26.183958 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:49:26.183958 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:26.183958 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:26.183958 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:26.183958 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:26.183958 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:26.183958 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:26.183958 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:26.183958 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:26.183958 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:26.183958 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:26.183958 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:26.183958 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:26.185332 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:26.185332 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:26.185332 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:26.185332 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:26.185332 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:49:26.185332 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:49:26.185332 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:49:26.185332 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:49:26.185332 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:49:26.185332 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:49:26.185332 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 04 00:49:26.185332 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:26.189008 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 513/1030] 10.222.0.27 () {56 vars in 1120 bytes} [Sun Sep 4 00:49:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:26.190225 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d511bf-d13c-41be-9004-d7771d54c6bf None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:26.255476 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cdcbe5-fd91-4866-ae8b-5dea160c2819 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:26.255759 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cdcbe5-fd91-4866-ae8b-5dea160c2819 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:26.256031 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cdcbe5-fd91-4866-ae8b-5dea160c2819 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:26.257515 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 514/1031] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 00:49:26.262618 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e0907-3af7-47ec-9667-d40ac8bfd276 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:26.262964 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e0907-3af7-47ec-9667-d40ac8bfd276 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:26.263243 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e0907-3af7-47ec-9667-d40ac8bfd276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:26.281901 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-523e0907-3af7-47ec-9667-d40ac8bfd276 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:26.283409 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-523e0907-3af7-47ec-9667-d40ac8bfd276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:26.309947 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d511bf-d13c-41be-9004-d7771d54c6bf None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:26.348109 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed082f6-36be-4490-8af6-685498b9349f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:26.348890 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed082f6-36be-4490-8af6-685498b9349f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:26.349179 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed082f6-36be-4490-8af6-685498b9349f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:26.349515 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed082f6-36be-4490-8af6-685498b9349f None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:26.350185 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed082f6-36be-4490-8af6-685498b9349f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:26.350492 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed082f6-36be-4490-8af6-685498b9349f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:26.351184 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:26.351881 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed082f6-36be-4490-8af6-685498b9349f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:26.366982 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 515/1032] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:26 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 00:49:26.367147 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 518/1033] 10.222.0.27 () {56 vars in 1115 bytes} [Sun Sep 4 00:49:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:26.367230 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e0907-3af7-47ec-9667-d40ac8bfd276 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:26.465745 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e0907-3af7-47ec-9667-d40ac8bfd276 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:26.501058 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26296dc6-4bd4-48dd-8c24-993aa3e8442d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:26.501685 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26296dc6-4bd4-48dd-8c24-993aa3e8442d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:26.501958 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26296dc6-4bd4-48dd-8c24-993aa3e8442d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:26.502222 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26296dc6-4bd4-48dd-8c24-993aa3e8442d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:26.512230 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26296dc6-4bd4-48dd-8c24-993aa3e8442d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:26.513692 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26296dc6-4bd4-48dd-8c24-993aa3e8442d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:26.528046 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-26296dc6-4bd4-48dd-8c24-993aa3e8442d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:49:26.528046 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:26.528046 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:26.528046 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:26.528046 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:26.528046 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:26.528046 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:26.528046 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:26.528046 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:26.528046 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:26.528046 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:26.528046 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:26.528046 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:49:26.528046 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:49:26.528046 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:49:26.528046 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:49:26.529262 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:26.529262 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:26.529262 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:26.529262 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:26.529262 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:26.529262 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:26.529262 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:26.529262 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:26.529262 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:26.529262 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:26.529262 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:26.529262 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:26.529262 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:26.529262 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:26.529262 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:26.529262 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:26.529262 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:49:26.530145 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:49:26.530145 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:49:26.530145 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:26.532848 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 516/1034] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:49:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:49:26.535661 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed082f6-36be-4490-8af6-685498b9349f None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:26.581802 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:26.624578 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26296dc6-4bd4-48dd-8c24-993aa3e8442d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:26.644694 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed082f6-36be-4490-8af6-685498b9349f None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:26.681351 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e95ab-dbc4-41b8-b8cc-d6d8edef1339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:26.682010 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e95ab-dbc4-41b8-b8cc-d6d8edef1339 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:26.682220 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e95ab-dbc4-41b8-b8cc-d6d8edef1339 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:26.682495 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e95ab-dbc4-41b8-b8cc-d6d8edef1339 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:26.683059 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:26.683178 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506e95ab-dbc4-41b8-b8cc-d6d8edef1339 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:26.683424 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506e95ab-dbc4-41b8-b8cc-d6d8edef1339 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:26.684842 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506e95ab-dbc4-41b8-b8cc-d6d8edef1339 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:26.699107 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 519/1035] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:49:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:26.705265 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e95ab-dbc4-41b8-b8cc-d6d8edef1339 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:26.735702 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26296dc6-4bd4-48dd-8c24-993aa3e8442d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:26.769042 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83f757-8735-49de-8071-5c68881700ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:26.769659 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83f757-8735-49de-8071-5c68881700ad None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:26.769935 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83f757-8735-49de-8071-5c68881700ad None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:26.770204 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83f757-8735-49de-8071-5c68881700ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:26.771946 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f83f757-8735-49de-8071-5c68881700ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:26.773394 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f83f757-8735-49de-8071-5c68881700ad None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:26.774985 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 517/1036] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:49:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:49:26.781174 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83f757-8735-49de-8071-5c68881700ad None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:26.820659 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e95ab-dbc4-41b8-b8cc-d6d8edef1339 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:26.822313 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:26.865871 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f028ad-e84b-4aa8-bed4-e18eaa321191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:26.866516 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f028ad-e84b-4aa8-bed4-e18eaa321191 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:26.866791 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f028ad-e84b-4aa8-bed4-e18eaa321191 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:26.867069 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f028ad-e84b-4aa8-bed4-e18eaa321191 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:26.875245 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f028ad-e84b-4aa8-bed4-e18eaa321191 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:26.877220 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f028ad-e84b-4aa8-bed4-e18eaa321191 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:26.888968 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-e2f028ad-e84b-4aa8-bed4-e18eaa321191 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 04 00:49:26.888968 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:26.888968 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:26.888968 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:26.888968 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:26.888968 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:26.888968 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:26.888968 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:26.888968 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:26.888968 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:26.888968 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:26.888968 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:26.888968 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:49:26.888968 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:49:26.888968 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:49:26.888968 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:49:26.890693 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:26.890693 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:26.890693 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:49:26.890693 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:49:26.890693 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:49:26.890693 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:49:26.890693 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:26.890693 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:26.890693 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:26.890693 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:26.890693 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:26.890693 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:26.890693 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:26.890693 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:26.890693 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:26.890693 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:26.890693 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:26.890693 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:26.891735 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:26.891735 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:26.891735 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:49:26.891735 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:49:26.891735 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:49:26.891735 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:49:26.891735 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:49:26.891735 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:49:26.891735 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 04 00:49:26.891735 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:26.892284 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83f757-8735-49de-8071-5c68881700ad None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:26.895094 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 520/1037] 10.222.0.27 () {56 vars in 1093 bytes} [Sun Sep 4 00:49:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:49:26.895961 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f028ad-e84b-4aa8-bed4-e18eaa321191 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:26.915455 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:26.930610 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc88703f-8cac-49e1-b34e-9870c1473b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:26.931222 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88703f-8cac-49e1-b34e-9870c1473b3d None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:26.931500 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88703f-8cac-49e1-b34e-9870c1473b3d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:26.931772 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88703f-8cac-49e1-b34e-9870c1473b3d None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:26.932419 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc88703f-8cac-49e1-b34e-9870c1473b3d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:26.933861 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc88703f-8cac-49e1-b34e-9870c1473b3d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:26.998660 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f028ad-e84b-4aa8-bed4-e18eaa321191 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:27.032284 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4741005a-0335-4d62-a723-a4d9424adeb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:27.032913 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741005a-0335-4d62-a723-a4d9424adeb3 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:27.033194 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741005a-0335-4d62-a723-a4d9424adeb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:27.033465 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741005a-0335-4d62-a723-a4d9424adeb3 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:27.034181 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4741005a-0335-4d62-a723-a4d9424adeb3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:27.034491 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4741005a-0335-4d62-a723-a4d9424adeb3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:27.035957 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4741005a-0335-4d62-a723-a4d9424adeb3 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:27.053420 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 521/1038] 10.222.0.27 () {56 vars in 1133 bytes} [Sun Sep 4 00:49:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:27.054613 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f257e66-b28a-4661-917f-2d9357bf0cdf None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:27.054902 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f257e66-b28a-4661-917f-2d9357bf0cdf None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:27.055179 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f257e66-b28a-4661-917f-2d9357bf0cdf None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:27.056591 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 522/1039] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:49:27.059237 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f257e66-b28a-4661-917f-2d9357bf0cdf None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:27.072595 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 518/1040] 10.222.0.27 () {60 vars in 1132 bytes} [Sun Sep 4 00:49:26 2022] POST /identity/v3/users => generated 285 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 00:49:27.072595 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2329af2-3624-428f-bd80-f20f53f84c65 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:27.072595 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2329af2-3624-428f-bd80-f20f53f84c65 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:27.072595 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2329af2-3624-428f-bd80-f20f53f84c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:27.084512 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:27.092296 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-e2329af2-3624-428f-bd80-f20f53f84c65 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:27.094417 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e2329af2-3624-428f-bd80-f20f53f84c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:27.170055 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 519/1041] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:27 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:49:27.180308 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f257e66-b28a-4661-917f-2d9357bf0cdf None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:27.221500 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87556fa-eb5e-42b9-93ed-ac3ca10cb62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:27.222120 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87556fa-eb5e-42b9-93ed-ac3ca10cb62b None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:27.222404 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87556fa-eb5e-42b9-93ed-ac3ca10cb62b None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:27.222683 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87556fa-eb5e-42b9-93ed-ac3ca10cb62b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:27.301083 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87556fa-eb5e-42b9-93ed-ac3ca10cb62b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:27.302918 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87556fa-eb5e-42b9-93ed-ac3ca10cb62b None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:27.324823 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:27.406788 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 523/1042] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:49:27 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 04 00:49:27.412479 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2329af2-3624-428f-bd80-f20f53f84c65 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:27.437190 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87556fa-eb5e-42b9-93ed-ac3ca10cb62b None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:27.522139 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2329af2-3624-428f-bd80-f20f53f84c65 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:27.530965 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:27.547167 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87556fa-eb5e-42b9-93ed-ac3ca10cb62b None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:27.558876 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f9ab66-60ab-42a5-be47-c2e7996293fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:27.559515 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f9ab66-60ab-42a5-be47-c2e7996293fb None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:27.559801 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f9ab66-60ab-42a5-be47-c2e7996293fb None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:27.560071 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f9ab66-60ab-42a5-be47-c2e7996293fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:27.571823 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f9ab66-60ab-42a5-be47-c2e7996293fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:27.572041 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:27.573324 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f9ab66-60ab-42a5-be47-c2e7996293fb None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:27.587321 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b832511-2898-42c4-9e80-866e19474c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:27.587884 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b832511-2898-42c4-9e80-866e19474c0d None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:27.588169 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b832511-2898-42c4-9e80-866e19474c0d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:27.588447 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b832511-2898-42c4-9e80-866e19474c0d None admin] PATH_INFO: `/v3/services/image` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:27.589074 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b832511-2898-42c4-9e80-866e19474c0d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:27.589174 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-e1f9ab66-60ab-42a5-be47-c2e7996293fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:49:27.589174 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:27.589174 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:27.589174 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:27.589174 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:27.589174 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:27.589174 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:27.589174 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:27.589174 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:27.589174 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:27.589174 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:27.589174 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:27.589174 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:49:27.589174 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:49:27.589174 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:49:27.589174 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:49:27.590832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:27.590832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:27.590832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:27.590832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:27.590832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:27.590832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:27.590832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:27.590832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:27.590832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:27.590832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:27.590832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:27.590832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:27.590832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:27.590832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:27.590832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:27.590832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:27.590832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:49:27.591742 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:49:27.591742 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:49:27.591742 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:27.591913 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b832511-2898-42c4-9e80-866e19474c0d None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:27.591986 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 520/1043] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:49:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:49:27.596878 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f9ab66-60ab-42a5-be47-c2e7996293fb None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:27.602973 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-4b832511-2898-42c4-9e80-866e19474c0d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 04 00:49:27.602973 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:27.602973 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 00:49:27.602973 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 00:49:27.602973 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 00:49:27.602973 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 00:49:27.602973 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 00:49:27.602973 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:49:27.602973 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 04 00:49:27.602973 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:27.602973 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 00:49:27.602973 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:27.602973 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:27.602973 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:27.602973 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:27.602973 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:27.602973 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:27.603927 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:27.603927 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:27.603927 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:27.603927 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:27.603927 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:27.603927 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:27.603927 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 00:49:27.603927 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 00:49:27.603927 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 00:49:27.603927 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 00:49:27.603927 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:27.603927 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:27.603927 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:27.603927 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:27.603927 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:27.603927 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:27.605406 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:27.605406 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:27.605406 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:27.605406 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:27.605406 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:27.605406 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:27.605406 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:27.605406 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:27.605406 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:27.605406 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:27.605406 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 00:49:27.605406 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:49:27.605406 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 04 00:49:27.605406 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:27.605406 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 524/1044] 10.222.0.27 () {56 vars in 1105 bytes} [Sun Sep 4 00:49:27 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:49:27.609990 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b832511-2898-42c4-9e80-866e19474c0d None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:27.711228 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f9ab66-60ab-42a5-be47-c2e7996293fb None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:27.725873 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b832511-2898-42c4-9e80-866e19474c0d None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:27.746233 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a2b03-a8b5-4d80-ba7f-627cea102fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:27.746837 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a2b03-a8b5-4d80-ba7f-627cea102fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:27.747412 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a2b03-a8b5-4d80-ba7f-627cea102fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:27.747412 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a2b03-a8b5-4d80-ba7f-627cea102fc5 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:27.748011 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a2b03-a8b5-4d80-ba7f-627cea102fc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:27.748374 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a2b03-a8b5-4d80-ba7f-627cea102fc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:27.749982 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a2b03-a8b5-4d80-ba7f-627cea102fc5 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:27.761839 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf21d44-5ff5-4c2d-98dc-031e2b89046b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:27.762444 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf21d44-5ff5-4c2d-98dc-031e2b89046b None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:27.762723 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf21d44-5ff5-4c2d-98dc-031e2b89046b None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:27.762992 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf21d44-5ff5-4c2d-98dc-031e2b89046b None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:27.763614 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf21d44-5ff5-4c2d-98dc-031e2b89046b None admin] RBAC: Adding query filter params (name=image) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:27.763915 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf21d44-5ff5-4c2d-98dc-031e2b89046b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:27.765876 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf21d44-5ff5-4c2d-98dc-031e2b89046b None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:27.767633 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 521/1045] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:49:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:49:27.768764 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a2b03-a8b5-4d80-ba7f-627cea102fc5 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:27.771546 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 525/1046] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:49:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:49:27.776969 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf21d44-5ff5-4c2d-98dc-031e2b89046b None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:27.793854 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:27.828995 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:27.884814 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a2b03-a8b5-4d80-ba7f-627cea102fc5 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:27.887794 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf21d44-5ff5-4c2d-98dc-031e2b89046b None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:27.921733 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c60cd2-95f3-4fba-97e1-5f0da3ca98fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:27.922363 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c60cd2-95f3-4fba-97e1-5f0da3ca98fe None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:27.922642 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c60cd2-95f3-4fba-97e1-5f0da3ca98fe None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:27.922914 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c60cd2-95f3-4fba-97e1-5f0da3ca98fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:27.928253 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d70dd8-6e04-40ee-b9be-7240912848af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:27.929234 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d70dd8-6e04-40ee-b9be-7240912848af None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:27.933581 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c60cd2-95f3-4fba-97e1-5f0da3ca98fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:27.933695 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d70dd8-6e04-40ee-b9be-7240912848af None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:27.933695 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d70dd8-6e04-40ee-b9be-7240912848af None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:27.933695 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d70dd8-6e04-40ee-b9be-7240912848af None admin] RBAC: Adding query filter params (type=image) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:27.933695 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d70dd8-6e04-40ee-b9be-7240912848af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:27.934005 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c60cd2-95f3-4fba-97e1-5f0da3ca98fe None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:27.934094 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d70dd8-6e04-40ee-b9be-7240912848af None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:27.943408 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-66c60cd2-95f3-4fba-97e1-5f0da3ca98fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:49:27.943408 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:27.943408 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:27.943408 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:27.943408 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:27.943408 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:27.943408 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:27.943408 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:27.943408 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:27.943408 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:27.943408 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:27.943408 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:27.943408 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:49:27.943408 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:49:27.943408 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:49:27.944478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:49:27.944478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:27.944478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:27.944478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:27.944478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:27.944478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:27.944478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:27.944478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:27.944478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:27.944478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:27.944478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:27.944478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:27.944478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:27.944478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:27.944478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:27.944478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:27.944478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:27.946747 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:49:27.946747 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:49:27.946747 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:49:27.946747 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:49:27.946747 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:49:27.946747 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:49:27.946747 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:49:27.946747 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:27.946747 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 522/1047] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:49:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:27.946747 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c60cd2-95f3-4fba-97e1-5f0da3ca98fe None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:27.947591 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 526/1048] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:49:27 2022] GET /identity/v3/services?type=image => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:49:27.953398 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d70dd8-6e04-40ee-b9be-7240912848af None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:28.028435 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:28.050874 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c60cd2-95f3-4fba-97e1-5f0da3ca98fe None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:28.058861 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d70dd8-6e04-40ee-b9be-7240912848af None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:28.073372 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:28.088360 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef9a6b-640b-4c19-b41c-b65cb99df6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:28.089028 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef9a6b-640b-4c19-b41c-b65cb99df6c6 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:28.089306 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef9a6b-640b-4c19-b41c-b65cb99df6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:28.089560 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef9a6b-640b-4c19-b41c-b65cb99df6c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:28.090223 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ef9a6b-640b-4c19-b41c-b65cb99df6c6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:28.091595 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ef9a6b-640b-4c19-b41c-b65cb99df6c6 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:28.095010 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e943e49-245e-4c37-bad9-3dd2caba32b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:28.095613 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e943e49-245e-4c37-bad9-3dd2caba32b4 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:28.095892 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e943e49-245e-4c37-bad9-3dd2caba32b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:28.096160 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e943e49-245e-4c37-bad9-3dd2caba32b4 None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:28.096873 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e943e49-245e-4c37-bad9-3dd2caba32b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:28.097176 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e943e49-245e-4c37-bad9-3dd2caba32b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:28.098596 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e943e49-245e-4c37-bad9-3dd2caba32b4 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:28.111528 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 527/1049] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:49:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:49:28.117508 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e943e49-245e-4c37-bad9-3dd2caba32b4 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:28.205621 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 523/1050] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 00:49:27 2022] POST /identity/v3/endpoints => generated 327 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:49:28.235924 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e943e49-245e-4c37-bad9-3dd2caba32b4 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:28.281416 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90bcbb5-3e2e-4f80-8be4-c3add9362cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:28.282077 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90bcbb5-3e2e-4f80-8be4-c3add9362cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:28.282363 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90bcbb5-3e2e-4f80-8be4-c3add9362cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:28.282640 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90bcbb5-3e2e-4f80-8be4-c3add9362cf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:28.283378 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90bcbb5-3e2e-4f80-8be4-c3add9362cf4 None admin] RBAC: Adding query filter params (role.id=91d340dbd1f142c5aa33ad640eca891c, scope.project.id=5100a8d316d243fb8f82e9044e1b3b06, user.id=68bc00bda5194cd8a7928f8c4df4ccd1) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:28.283692 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90bcbb5-3e2e-4f80-8be4-c3add9362cf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:28.285182 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90bcbb5-3e2e-4f80-8be4-c3add9362cf4 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:28.291961 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 528/1051] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:49:28 2022] GET /identity/v3/role_assignments?user.id=68bc00bda5194cd8a7928f8c4df4ccd1&scope.project.id=5100a8d316d243fb8f82e9044e1b3b06&role.id=91d340dbd1f142c5aa33ad640eca891c => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:49:29.083556 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494b86d-3969-4672-9962-f6ecad3052e9 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:29.083864 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494b86d-3969-4672-9962-f6ecad3052e9 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:29.084223 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494b86d-3969-4672-9962-f6ecad3052e9 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:29.085719 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 524/1052] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:49:29.095670 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97230f99-4a72-43b3-bfea-dca9b3e8df67 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:29.096173 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97230f99-4a72-43b3-bfea-dca9b3e8df67 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:29.096285 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97230f99-4a72-43b3-bfea-dca9b3e8df67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:29.116787 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-97230f99-4a72-43b3-bfea-dca9b3e8df67 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:29.117998 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-97230f99-4a72-43b3-bfea-dca9b3e8df67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:29.145391 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:29.192503 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 529/1053] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 00:49:29.477675 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7494b86d-3969-4672-9962-f6ecad3052e9 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:29.590010 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7494b86d-3969-4672-9962-f6ecad3052e9 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:29.630284 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa50af-5faa-4219-a95e-e5ca9ace2139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:29.631254 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa50af-5faa-4219-a95e-e5ca9ace2139 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:29.631752 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa50af-5faa-4219-a95e-e5ca9ace2139 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:29.632158 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa50af-5faa-4219-a95e-e5ca9ace2139 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:29.634849 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:29.646607 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fa50af-5faa-4219-a95e-e5ca9ace2139 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:49:29.651732 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-d3fa50af-5faa-4219-a95e-e5ca9ace2139 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:49:29.651732 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:29.651732 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:29.651732 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:29.651732 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:29.651732 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:29.651732 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:29.651732 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:29.651732 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:29.651732 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:29.651732 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:29.651732 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:29.651732 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:49:29.651732 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:49:29.651732 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:49:29.651732 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:49:29.659909 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:49:29.659909 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:49:29.659909 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:49:29.659909 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:49:29.659909 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:29.659909 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:29.659909 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:29.659909 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:29.659909 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:29.659909 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:29.659909 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:29.659909 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:29.659909 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:29.659909 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:29.659909 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:29.659909 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:29.661147 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:29.661147 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:29.661147 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:29.661147 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:29.661147 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:49:29.661147 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:49:29.661147 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:49:29.661147 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:49:29.661147 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:49:29.661147 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:49:29.661147 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:49:29.661147 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:29.661147 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 525/1054] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:49:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:29.661841 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97230f99-4a72-43b3-bfea-dca9b3e8df67 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:29.770136 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97230f99-4a72-43b3-bfea-dca9b3e8df67 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:29.807109 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd45406-308e-457b-8391-c8f85927399e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:29.807726 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd45406-308e-457b-8391-c8f85927399e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:29.808009 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd45406-308e-457b-8391-c8f85927399e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:29.808285 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd45406-308e-457b-8391-c8f85927399e None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:29.808992 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd45406-308e-457b-8391-c8f85927399e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:29.809299 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd45406-308e-457b-8391-c8f85927399e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:29.810716 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd45406-308e-457b-8391-c8f85927399e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:29.820137 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 530/1055] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:49:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:29.826379 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa50af-5faa-4219-a95e-e5ca9ace2139 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:29.942116 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa50af-5faa-4219-a95e-e5ca9ace2139 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:29.978518 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972bb617-56f9-4ad1-8fab-4d715872dae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:29.979160 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bb617-56f9-4ad1-8fab-4d715872dae5 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:29.979437 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bb617-56f9-4ad1-8fab-4d715872dae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:29.979708 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bb617-56f9-4ad1-8fab-4d715872dae5 None admin] PATH_INFO: `/v3/users/42c7e11f1b254c58aa492374034f0900` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:29.995790 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972bb617-56f9-4ad1-8fab-4d715872dae5 None admin] RBAC: Authorizing `identity:get_user(user_id=42c7e11f1b254c58aa492374034f0900)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:29.997277 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972bb617-56f9-4ad1-8fab-4d715872dae5 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:29.999765 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 526/1056] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:49:29 2022] GET /identity/v3/users/42c7e11f1b254c58aa492374034f0900 => generated 285 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:30.005995 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd45406-308e-457b-8391-c8f85927399e None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:30.040307 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:30.131600 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd45406-308e-457b-8391-c8f85927399e None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:30.166321 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade9ae0d-db02-4b2a-9efb-f12bfff5c48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:30.166926 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9ae0d-db02-4b2a-9efb-f12bfff5c48b None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:30.167206 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9ae0d-db02-4b2a-9efb-f12bfff5c48b None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:30.167478 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9ae0d-db02-4b2a-9efb-f12bfff5c48b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:30.235433 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43fd20-ffe6-4042-a775-f900ca7b680f None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:30.235711 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43fd20-ffe6-4042-a775-f900ca7b680f None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:30.235984 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43fd20-ffe6-4042-a775-f900ca7b680f None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:30.237605 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 527/1057] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 00:49:30.242243 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a944dd-f182-4518-8292-cd292c5a0505 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:30.242537 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a944dd-f182-4518-8292-cd292c5a0505 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:30.242818 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a944dd-f182-4518-8292-cd292c5a0505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:30.244353 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade9ae0d-db02-4b2a-9efb-f12bfff5c48b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:30.245867 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade9ae0d-db02-4b2a-9efb-f12bfff5c48b None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:30.261583 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f2a944dd-f182-4518-8292-cd292c5a0505 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:30.262787 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f2a944dd-f182-4518-8292-cd292c5a0505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:30.274232 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:30.316396 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:30.336169 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 528/1058] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 00:49:30.337437 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46901ee-fe42-41f3-9b62-65d8c0b94ac6 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:30.337721 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46901ee-fe42-41f3-9b62-65d8c0b94ac6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:30.337995 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46901ee-fe42-41f3-9b62-65d8c0b94ac6 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:30.340524 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 529/1059] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:49:30.344580 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d73d80-169d-440b-a89c-bd0f6dc8abb6 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:30.345038 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d73d80-169d-440b-a89c-bd0f6dc8abb6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:30.345442 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d73d80-169d-440b-a89c-bd0f6dc8abb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:30.352524 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 531/1060] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:49:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 04 00:49:30.358573 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade9ae0d-db02-4b2a-9efb-f12bfff5c48b None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:30.367865 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-62d73d80-169d-440b-a89c-bd0f6dc8abb6 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:30.369074 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-62d73d80-169d-440b-a89c-bd0f6dc8abb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:30.435564 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 530/1061] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 00:49:30.463049 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade9ae0d-db02-4b2a-9efb-f12bfff5c48b None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:30.486448 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:30.497223 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b3a4d0-be46-4fe7-872f-cbcc922bc997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:30.497848 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3a4d0-be46-4fe7-872f-cbcc922bc997 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:30.498130 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3a4d0-be46-4fe7-872f-cbcc922bc997 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:30.498515 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3a4d0-be46-4fe7-872f-cbcc922bc997 None admin] PATH_INFO: `/v3/projects/fd852c22be454accb5dd60e3b6e60ef7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:30.500233 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b3a4d0-be46-4fe7-872f-cbcc922bc997 None admin] RBAC: Authorizing `identity:get_project(project_id=fd852c22be454accb5dd60e3b6e60ef7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:30.501705 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b3a4d0-be46-4fe7-872f-cbcc922bc997 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:30.508864 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 532/1062] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:49:30 2022] GET /identity/v3/projects/fd852c22be454accb5dd60e3b6e60ef7 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:49:30.509577 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d73d80-169d-440b-a89c-bd0f6dc8abb6 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:30.611345 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d73d80-169d-440b-a89c-bd0f6dc8abb6 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:30.613726 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b3a4d0-be46-4fe7-872f-cbcc922bc997 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:30.625138 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:30.647227 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e9691-6e03-4dd3-a58d-2efb25c5ae90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:30.647833 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e9691-6e03-4dd3-a58d-2efb25c5ae90 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:30.648130 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e9691-6e03-4dd3-a58d-2efb25c5ae90 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:30.648392 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e9691-6e03-4dd3-a58d-2efb25c5ae90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:30.649145 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37e9691-6e03-4dd3-a58d-2efb25c5ae90 None admin] RBAC: Adding query filter params (role.id=c8a59d34ee2f4cefa6940cac778260c7, scope.project.id=fd852c22be454accb5dd60e3b6e60ef7, user.id=42c7e11f1b254c58aa492374034f0900) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:30.649455 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37e9691-6e03-4dd3-a58d-2efb25c5ae90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:30.650877 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37e9691-6e03-4dd3-a58d-2efb25c5ae90 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:30.664736 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 531/1063] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:49:30 2022] GET /identity/v3/role_assignments?user.id=42c7e11f1b254c58aa492374034f0900&scope.project.id=fd852c22be454accb5dd60e3b6e60ef7&role.id=c8a59d34ee2f4cefa6940cac778260c7 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:49:30.708408 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e9691-6e03-4dd3-a58d-2efb25c5ae90 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:30.738078 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b3a4d0-be46-4fe7-872f-cbcc922bc997 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:30.784191 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe13414-dd1b-4aa4-a90f-58b55e16de84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:30.785702 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe13414-dd1b-4aa4-a90f-58b55e16de84 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:30.785983 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe13414-dd1b-4aa4-a90f-58b55e16de84 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:30.786255 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe13414-dd1b-4aa4-a90f-58b55e16de84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:30.817388 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e9691-6e03-4dd3-a58d-2efb25c5ae90 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:30.854080 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee5753a-a662-4dd9-b26b-3e3b01738301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:30.854972 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5753a-a662-4dd9-b26b-3e3b01738301 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:30.855252 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5753a-a662-4dd9-b26b-3e3b01738301 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:30.855520 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5753a-a662-4dd9-b26b-3e3b01738301 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:30.860647 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:30.868722 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee5753a-a662-4dd9-b26b-3e3b01738301 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:49:30.870538 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe13414-dd1b-4aa4-a90f-58b55e16de84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:30.872140 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe13414-dd1b-4aa4-a90f-58b55e16de84 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:30.873329 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-1ee5753a-a662-4dd9-b26b-3e3b01738301 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 04 00:49:30.873329 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:30.873329 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:30.873329 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:30.873329 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:30.873329 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:30.873329 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:30.873329 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:30.873329 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:30.873329 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:30.873329 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:30.873329 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:30.873329 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:49:30.873329 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:49:30.873329 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:49:30.873329 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:49:30.874616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:49:30.874616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:49:30.874616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:49:30.874616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:49:30.874616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:30.874616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:30.874616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:30.874616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:30.874616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:30.874616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:30.874616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:30.874616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:30.874616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:30.874616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:30.874616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:30.874616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:30.875705 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:30.875705 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:30.875705 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:30.875705 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:30.875705 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:49:30.875705 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:49:30.875705 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:49:30.875705 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:49:30.875705 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:49:30.875705 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:49:30.875705 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 04 00:49:30.875705 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:30.879712 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 532/1064] 10.222.0.27 () {56 vars in 1120 bytes} [Sun Sep 4 00:49:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:49:30.880608 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee5753a-a662-4dd9-b26b-3e3b01738301 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:30.919211 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:30.980999 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 533/1065] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:49:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 04 00:49:30.987161 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe13414-dd1b-4aa4-a90f-58b55e16de84 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:30.988911 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee5753a-a662-4dd9-b26b-3e3b01738301 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:31.028389 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf5bfe-ae39-4fb0-9261-9b2ab43c62a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:31.029186 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf5bfe-ae39-4fb0-9261-9b2ab43c62a9 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:31.029873 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf5bfe-ae39-4fb0-9261-9b2ab43c62a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:31.030881 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf5bfe-ae39-4fb0-9261-9b2ab43c62a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:31.031852 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bf5bfe-ae39-4fb0-9261-9b2ab43c62a9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:31.032263 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bf5bfe-ae39-4fb0-9261-9b2ab43c62a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:31.033805 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bf5bfe-ae39-4fb0-9261-9b2ab43c62a9 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:31.047949 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 533/1066] 10.222.0.27 () {56 vars in 1115 bytes} [Sun Sep 4 00:49:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:31.049660 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf5bfe-ae39-4fb0-9261-9b2ab43c62a9 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:31.080471 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:31.089286 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe13414-dd1b-4aa4-a90f-58b55e16de84 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:31.126738 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018c3eb-2625-44ed-b195-4fe82f8d9bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:31.127402 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018c3eb-2625-44ed-b195-4fe82f8d9bdc None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:31.127753 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018c3eb-2625-44ed-b195-4fe82f8d9bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:31.128099 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018c3eb-2625-44ed-b195-4fe82f8d9bdc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:31.138662 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e018c3eb-2625-44ed-b195-4fe82f8d9bdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:31.140544 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e018c3eb-2625-44ed-b195-4fe82f8d9bdc None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:31.155857 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:31.156027 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-e018c3eb-2625-44ed-b195-4fe82f8d9bdc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:49:31.156027 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:31.156027 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:31.156027 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:31.156027 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:31.156027 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:31.156027 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:31.156027 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:31.156027 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:31.156027 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:31.156027 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:31.156027 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:31.156027 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:49:31.156027 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:49:31.156027 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:49:31.156027 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:49:31.157326 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:31.157326 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:31.157326 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:31.157326 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:31.157326 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:31.157326 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:31.157326 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:31.157326 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:31.157326 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:31.157326 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:31.157326 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:31.157326 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:31.157326 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:31.157326 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:31.157326 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:31.157326 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:31.157326 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:49:31.158599 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:49:31.158599 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:49:31.158599 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:31.162149 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 534/1067] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:49:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:49:31.162807 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018c3eb-2625-44ed-b195-4fe82f8d9bdc None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:31.163263 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf5bfe-ae39-4fb0-9261-9b2ab43c62a9 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:31.200784 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd08fb0-e406-4174-8564-9b4daf18875f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:31.201412 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd08fb0-e406-4174-8564-9b4daf18875f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:31.201698 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd08fb0-e406-4174-8564-9b4daf18875f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:31.202470 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd08fb0-e406-4174-8564-9b4daf18875f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:31.213838 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd08fb0-e406-4174-8564-9b4daf18875f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:31.215533 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd08fb0-e406-4174-8564-9b4daf18875f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:31.231419 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-cfd08fb0-e406-4174-8564-9b4daf18875f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:49:31.231419 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:31.231419 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:31.231419 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:31.231419 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:31.231419 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:31.231419 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:31.231419 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:31.231419 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:31.231419 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:31.231419 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:31.231419 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:31.231419 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:49:31.231419 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:49:31.231419 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:49:31.231419 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:49:31.232444 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:31.232444 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:31.232444 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:31.232444 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:31.232444 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:31.232444 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:31.232444 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:31.232444 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:31.232444 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:31.232444 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:31.232444 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:31.232444 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:31.232444 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:31.232444 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:31.232444 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:31.232444 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:31.232444 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:49:31.233435 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:49:31.233435 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:49:31.233435 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:31.235450 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 534/1068] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:49:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:49:31.241010 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd08fb0-e406-4174-8564-9b4daf18875f None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:31.279765 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018c3eb-2625-44ed-b195-4fe82f8d9bdc None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:31.316833 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd32a79-dbd1-4370-b46d-2f1eece86f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:31.317439 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd32a79-dbd1-4370-b46d-2f1eece86f0c None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:31.317718 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd32a79-dbd1-4370-b46d-2f1eece86f0c None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:31.317990 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd32a79-dbd1-4370-b46d-2f1eece86f0c None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:31.318626 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd32a79-dbd1-4370-b46d-2f1eece86f0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:31.318925 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd32a79-dbd1-4370-b46d-2f1eece86f0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:31.320291 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd32a79-dbd1-4370-b46d-2f1eece86f0c None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:31.340493 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 535/1069] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:49:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:49:31.356015 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd08fb0-e406-4174-8564-9b4daf18875f None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:31.394099 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ca006-83b8-424a-8494-965a2a49265c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:31.394626 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ca006-83b8-424a-8494-965a2a49265c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:31.394905 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ca006-83b8-424a-8494-965a2a49265c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:31.395177 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ca006-83b8-424a-8494-965a2a49265c None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:31.395824 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6ca006-83b8-424a-8494-965a2a49265c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:31.396124 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6ca006-83b8-424a-8494-965a2a49265c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:31.397549 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6ca006-83b8-424a-8494-965a2a49265c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:31.411363 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 535/1070] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:49:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:49:31.417735 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd32a79-dbd1-4370-b46d-2f1eece86f0c None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:31.421763 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:31.527971 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd32a79-dbd1-4370-b46d-2f1eece86f0c None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:31.566763 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f687d3-c3b9-4b30-aadc-e381f4c64a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:31.567657 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f687d3-c3b9-4b30-aadc-e381f4c64a4f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:31.568094 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f687d3-c3b9-4b30-aadc-e381f4c64a4f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:31.568371 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f687d3-c3b9-4b30-aadc-e381f4c64a4f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:31.576203 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f687d3-c3b9-4b30-aadc-e381f4c64a4f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:31.578793 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f687d3-c3b9-4b30-aadc-e381f4c64a4f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:31.590476 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-e2f687d3-c3b9-4b30-aadc-e381f4c64a4f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 04 00:49:31.590476 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:31.590476 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:31.590476 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:31.590476 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:31.590476 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:31.590476 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:31.590476 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:31.590476 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:31.590476 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:31.590476 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:31.590476 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:31.590476 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:49:31.590476 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:49:31.590476 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:49:31.590476 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:49:31.591623 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:31.591623 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:31.591623 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:49:31.591623 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:49:31.591623 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:49:31.591623 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:49:31.591623 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:31.591623 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:31.591623 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:31.591623 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:31.591623 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:31.591623 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:31.591623 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:31.591623 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:31.591623 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:31.591623 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:31.591623 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:31.591623 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:31.593527 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:31.593527 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:31.593527 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:49:31.593527 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:49:31.593527 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:49:31.593527 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:49:31.593527 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:49:31.593527 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:49:31.593527 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 04 00:49:31.593527 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:31.593527 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 536/1071] 10.222.0.27 () {56 vars in 1093 bytes} [Sun Sep 4 00:49:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:31.599177 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ca006-83b8-424a-8494-965a2a49265c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:31.614302 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:31.705505 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ca006-83b8-424a-8494-965a2a49265c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:31.741018 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9108e56a-a0de-4bbf-b43c-5b61803ac256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:31.741620 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108e56a-a0de-4bbf-b43c-5b61803ac256 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:31.741896 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108e56a-a0de-4bbf-b43c-5b61803ac256 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:31.742165 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108e56a-a0de-4bbf-b43c-5b61803ac256 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:31.742881 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9108e56a-a0de-4bbf-b43c-5b61803ac256 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:31.743183 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9108e56a-a0de-4bbf-b43c-5b61803ac256 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:31.744632 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9108e56a-a0de-4bbf-b43c-5b61803ac256 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:31.762690 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 536/1072] 10.222.0.27 () {56 vars in 1133 bytes} [Sun Sep 4 00:49:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:31.768971 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f687d3-c3b9-4b30-aadc-e381f4c64a4f None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:31.832900 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:31.871906 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f687d3-c3b9-4b30-aadc-e381f4c64a4f None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:31.906429 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b92df2-3973-49da-8797-e91c484d7d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:31.907056 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b92df2-3973-49da-8797-e91c484d7d1d None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:31.907341 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b92df2-3973-49da-8797-e91c484d7d1d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:31.907616 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b92df2-3973-49da-8797-e91c484d7d1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:31.918262 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b92df2-3973-49da-8797-e91c484d7d1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:31.919708 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b92df2-3973-49da-8797-e91c484d7d1d None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:31.933628 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-f2b92df2-3973-49da-8797-e91c484d7d1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:49:31.933628 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:31.933628 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:31.933628 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:31.933628 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:31.933628 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:31.933628 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:31.933628 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:31.933628 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:31.933628 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:31.933628 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:31.933628 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:31.933628 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:49:31.933628 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:49:31.933628 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:49:31.933628 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:49:31.934739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:31.934739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:31.934739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:31.934739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:31.934739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:31.934739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:31.934739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:31.934739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:31.934739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:31.934739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:31.934739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:31.934739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:31.934739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:31.934739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:31.934739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:31.934739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:31.934739 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:49:31.935842 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:49:31.935842 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:49:31.935842 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:31.937130 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 537/1073] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:49:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:31.941034 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9108e56a-a0de-4bbf-b43c-5b61803ac256 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:32.022928 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:32.056543 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9108e56a-a0de-4bbf-b43c-5b61803ac256 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:32.089853 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c0466-aaba-4a72-ac05-8f4843e400a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:32.090453 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c0466-aaba-4a72-ac05-8f4843e400a8 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:32.090735 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c0466-aaba-4a72-ac05-8f4843e400a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:32.091009 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c0466-aaba-4a72-ac05-8f4843e400a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:32.091639 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c0466-aaba-4a72-ac05-8f4843e400a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:32.091945 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c0466-aaba-4a72-ac05-8f4843e400a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:32.093399 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c0466-aaba-4a72-ac05-8f4843e400a8 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:32.104979 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 537/1074] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:49:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:32.111114 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b92df2-3973-49da-8797-e91c484d7d1d None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:32.210700 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b92df2-3973-49da-8797-e91c484d7d1d None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:32.230692 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:32.249753 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d65acc-7df6-4085-993d-93520a72810e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:32.250385 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d65acc-7df6-4085-993d-93520a72810e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:32.250672 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d65acc-7df6-4085-993d-93520a72810e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:32.250946 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d65acc-7df6-4085-993d-93520a72810e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:32.257354 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d65acc-7df6-4085-993d-93520a72810e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:32.258830 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d65acc-7df6-4085-993d-93520a72810e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:32.270163 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-a6d65acc-7df6-4085-993d-93520a72810e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:49:32.270163 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:32.270163 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:32.270163 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:32.270163 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:32.270163 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:32.270163 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:32.270163 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:32.270163 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:32.270163 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:32.270163 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:32.270163 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:32.270163 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:49:32.270163 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:49:32.270163 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:49:32.271045 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:49:32.271045 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:32.271045 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:32.271045 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:32.271045 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:32.271045 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:32.271045 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:32.271045 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:32.271045 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:32.271045 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:32.271045 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:32.271045 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:32.271045 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:32.271045 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:32.271045 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:32.271045 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:32.271045 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:32.271920 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:49:32.271920 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:49:32.271920 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:49:32.271920 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:49:32.271920 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:49:32.271920 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:49:32.271920 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:49:32.271920 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:32.272616 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 538/1075] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:49:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:32.277052 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c0466-aaba-4a72-ac05-8f4843e400a8 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:32.374580 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c0466-aaba-4a72-ac05-8f4843e400a8 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:32.399543 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:32.407824 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7314738-41b9-4054-9686-780d1eb77f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:32.408428 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7314738-41b9-4054-9686-780d1eb77f0a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:32.408741 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7314738-41b9-4054-9686-780d1eb77f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:32.409010 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7314738-41b9-4054-9686-780d1eb77f0a None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:32.413938 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7314738-41b9-4054-9686-780d1eb77f0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:32.413938 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7314738-41b9-4054-9686-780d1eb77f0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:32.413938 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7314738-41b9-4054-9686-780d1eb77f0a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:32.424051 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 538/1076] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:49:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:32.430002 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d65acc-7df6-4085-993d-93520a72810e None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:32.527988 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d65acc-7df6-4085-993d-93520a72810e None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:32.561042 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479a64a-676a-4396-ab00-4fb00a310607 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:32.561382 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479a64a-676a-4396-ab00-4fb00a310607 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:32.561771 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479a64a-676a-4396-ab00-4fb00a310607 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:32.563074 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209aabad-a8bc-4748-bc3c-1f30a504e360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:32.563256 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 539/1077] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 00:49:32.563723 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209aabad-a8bc-4748-bc3c-1f30a504e360 None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:32.564001 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209aabad-a8bc-4748-bc3c-1f30a504e360 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:32.564270 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209aabad-a8bc-4748-bc3c-1f30a504e360 None admin] PATH_INFO: `/v3/projects/5100a8d316d243fb8f82e9044e1b3b06/users/68bc00bda5194cd8a7928f8c4df4ccd1/roles/91d340dbd1f142c5aa33ad640eca891c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:32.566377 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209aabad-a8bc-4748-bc3c-1f30a504e360 None admin] RBAC: Authorizing `identity:create_grant(project_id=5100a8d316d243fb8f82e9044e1b3b06, user_id=68bc00bda5194cd8a7928f8c4df4ccd1, role_id=91d340dbd1f142c5aa33ad640eca891c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:32.567985 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52796d24-a183-44cb-ba1d-3cf842ca9121 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:32.568095 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209aabad-a8bc-4748-bc3c-1f30a504e360 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:32.568269 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52796d24-a183-44cb-ba1d-3cf842ca9121 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:32.568574 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52796d24-a183-44cb-ba1d-3cf842ca9121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:32.586939 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-52796d24-a183-44cb-ba1d-3cf842ca9121 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:32.588133 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-52796d24-a183-44cb-ba1d-3cf842ca9121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:32.637592 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 539/1078] 10.222.0.27 () {58 vars in 1439 bytes} [Sun Sep 4 00:49:32 2022] PUT /identity/v3/projects/5100a8d316d243fb8f82e9044e1b3b06/users/68bc00bda5194cd8a7928f8c4df4ccd1/roles/91d340dbd1f142c5aa33ad640eca891c => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 00:49:32.655706 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 540/1079] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 00:49:32.931492 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209aabad-a8bc-4748-bc3c-1f30a504e360 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:32.967463 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:33.033606 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209aabad-a8bc-4748-bc3c-1f30a504e360 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:33.068274 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc7223-99cb-4819-8910-dfa5922eaf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:33.068957 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc7223-99cb-4819-8910-dfa5922eaf7f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:33.069240 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc7223-99cb-4819-8910-dfa5922eaf7f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:33.069514 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc7223-99cb-4819-8910-dfa5922eaf7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:33.080176 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fc7223-99cb-4819-8910-dfa5922eaf7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:49:33.084938 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-67fc7223-99cb-4819-8910-dfa5922eaf7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:49:33.084938 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:33.084938 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:33.084938 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:33.084938 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:33.084938 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:33.084938 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:33.084938 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:33.084938 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:33.084938 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:33.084938 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:33.084938 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:33.084938 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:49:33.084938 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:49:33.084938 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:49:33.084938 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:49:33.085909 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:49:33.085909 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:49:33.085909 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:49:33.085909 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:49:33.085909 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:33.085909 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:33.085909 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:33.085909 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:33.085909 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:33.085909 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:33.085909 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:33.085909 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:33.085909 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:33.085909 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:33.085909 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:33.085909 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:33.086772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:33.086772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:33.086772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:33.086772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:33.086772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:49:33.086772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:49:33.086772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:49:33.086772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:49:33.086772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:49:33.086772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:49:33.086772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:49:33.086772 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:33.086772 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 540/1080] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:49:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:33.092105 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52796d24-a183-44cb-ba1d-3cf842ca9121 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:33.192084 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52796d24-a183-44cb-ba1d-3cf842ca9121 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:33.208972 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:33.226199 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3ec25-2725-4c26-b410-3fe6a3ce6328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:33.226814 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3ec25-2725-4c26-b410-3fe6a3ce6328 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:33.227094 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3ec25-2725-4c26-b410-3fe6a3ce6328 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:33.227368 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3ec25-2725-4c26-b410-3fe6a3ce6328 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:33.229278 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d3ec25-2725-4c26-b410-3fe6a3ce6328 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:33.229278 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d3ec25-2725-4c26-b410-3fe6a3ce6328 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:33.229972 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d3ec25-2725-4c26-b410-3fe6a3ce6328 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:33.239903 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 541/1081] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:49:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:33.245601 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc7223-99cb-4819-8910-dfa5922eaf7f None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:33.339035 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:33.345726 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc7223-99cb-4819-8910-dfa5922eaf7f None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:33.380202 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a36e5-1057-419f-9dc3-a63c4e5b12af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:33.380863 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a36e5-1057-419f-9dc3-a63c4e5b12af None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:33.381147 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a36e5-1057-419f-9dc3-a63c4e5b12af None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:33.381422 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a36e5-1057-419f-9dc3-a63c4e5b12af None admin] PATH_INFO: `/v3/users/42c7e11f1b254c58aa492374034f0900` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:33.382532 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28a36e5-1057-419f-9dc3-a63c4e5b12af None admin] RBAC: Authorizing `identity:get_user(user_id=42c7e11f1b254c58aa492374034f0900)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:33.383967 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28a36e5-1057-419f-9dc3-a63c4e5b12af None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:33.385571 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 541/1082] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:49:33 2022] GET /identity/v3/users/42c7e11f1b254c58aa492374034f0900 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:33.391327 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3ec25-2725-4c26-b410-3fe6a3ce6328 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:33.489197 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3ec25-2725-4c26-b410-3fe6a3ce6328 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:33.521814 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54cb32-de7f-4a04-80f8-c8716de15462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:33.522429 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54cb32-de7f-4a04-80f8-c8716de15462 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:33.522736 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54cb32-de7f-4a04-80f8-c8716de15462 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:33.523011 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54cb32-de7f-4a04-80f8-c8716de15462 None admin] PATH_INFO: `/v3/projects/fd852c22be454accb5dd60e3b6e60ef7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:33.524107 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54cb32-de7f-4a04-80f8-c8716de15462 None admin] RBAC: Authorizing `identity:get_project(project_id=fd852c22be454accb5dd60e3b6e60ef7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:33.525573 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54cb32-de7f-4a04-80f8-c8716de15462 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:33.527193 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 542/1083] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:49:33 2022] GET /identity/v3/projects/fd852c22be454accb5dd60e3b6e60ef7 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:33.533739 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a36e5-1057-419f-9dc3-a63c4e5b12af None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:33.631447 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a36e5-1057-419f-9dc3-a63c4e5b12af None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:33.665650 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc75530-a16e-4556-9eb1-612a050c968b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:33.666298 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc75530-a16e-4556-9eb1-612a050c968b None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:33.666578 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc75530-a16e-4556-9eb1-612a050c968b None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:33.666852 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc75530-a16e-4556-9eb1-612a050c968b None admin] PATH_INFO: `/v3/projects/fd852c22be454accb5dd60e3b6e60ef7/users/42c7e11f1b254c58aa492374034f0900/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:33.668745 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc75530-a16e-4556-9eb1-612a050c968b None admin] RBAC: Authorizing `identity:create_grant(project_id=fd852c22be454accb5dd60e3b6e60ef7, user_id=42c7e11f1b254c58aa492374034f0900, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:33.670203 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc75530-a16e-4556-9eb1-612a050c968b None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:33.781478 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 542/1084] 10.222.0.27 () {58 vars in 1439 bytes} [Sun Sep 4 00:49:33 2022] PUT /identity/v3/projects/fd852c22be454accb5dd60e3b6e60ef7/users/42c7e11f1b254c58aa492374034f0900/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 00:49:34.531404 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0fdbfa-b54c-403f-8e9d-ddb5f6daa7f9 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:34.531944 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0fdbfa-b54c-403f-8e9d-ddb5f6daa7f9 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:34.531944 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0fdbfa-b54c-403f-8e9d-ddb5f6daa7f9 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:34.533317 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 543/1085] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:49:34.538865 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25ac59-cba3-414e-8415-d90630113757 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:34.539255 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25ac59-cba3-414e-8415-d90630113757 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:34.539422 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25ac59-cba3-414e-8415-d90630113757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:34.557555 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-4d25ac59-cba3-414e-8415-d90630113757 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:34.558752 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-4d25ac59-cba3-414e-8415-d90630113757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:34.575653 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:34.625292 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 543/1086] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:49:34.897111 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0fdbfa-b54c-403f-8e9d-ddb5f6daa7f9 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:34.948780 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:35.004029 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0fdbfa-b54c-403f-8e9d-ddb5f6daa7f9 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:35.037724 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f5ddd-f3b6-4509-b872-77b621e25a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:35.038607 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f5ddd-f3b6-4509-b872-77b621e25a81 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:35.038888 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f5ddd-f3b6-4509-b872-77b621e25a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:35.039168 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f5ddd-f3b6-4509-b872-77b621e25a81 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:35.048992 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f5ddd-f3b6-4509-b872-77b621e25a81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:49:35.053755 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-0d3f5ddd-f3b6-4509-b872-77b621e25a81 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 04 00:49:35.053755 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:35.053755 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:35.053755 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:35.053755 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:35.053755 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:35.053755 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:35.053755 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:35.053755 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:35.053755 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:35.053755 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:35.053755 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:35.053755 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:49:35.053755 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:49:35.053755 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:49:35.053755 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:49:35.054850 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:49:35.054850 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:49:35.054850 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:49:35.054850 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:49:35.054850 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:35.054850 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:35.054850 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:35.054850 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:35.054850 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:35.054850 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:35.054850 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:35.054850 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:35.054850 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:35.054850 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:35.054850 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:35.054850 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:35.055684 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:35.055684 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:35.055684 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:35.055684 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:35.055684 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:49:35.055684 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:49:35.055684 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:49:35.055684 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:49:35.055684 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:49:35.055684 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:49:35.055684 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 04 00:49:35.055684 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:35.055684 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 544/1087] 10.222.0.27 () {56 vars in 1120 bytes} [Sun Sep 4 00:49:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:35.060814 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25ac59-cba3-414e-8415-d90630113757 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:35.158411 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25ac59-cba3-414e-8415-d90630113757 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:35.191814 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9fcded-a709-474f-b742-4e78aea31e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:35.192416 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9fcded-a709-474f-b742-4e78aea31e61 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:35.192723 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9fcded-a709-474f-b742-4e78aea31e61 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:35.192995 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9fcded-a709-474f-b742-4e78aea31e61 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:35.193651 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9fcded-a709-474f-b742-4e78aea31e61 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:35.193953 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9fcded-a709-474f-b742-4e78aea31e61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:35.195317 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9fcded-a709-474f-b742-4e78aea31e61 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:35.208215 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 544/1088] 10.222.0.27 () {56 vars in 1115 bytes} [Sun Sep 4 00:49:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:35.209385 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f5ddd-f3b6-4509-b872-77b621e25a81 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:35.310727 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f5ddd-f3b6-4509-b872-77b621e25a81 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:35.319831 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:35.344876 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21829d36-2c9b-4c95-95b5-218fbfdb5e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:35.349521 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21829d36-2c9b-4c95-95b5-218fbfdb5e65 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:35.349521 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21829d36-2c9b-4c95-95b5-218fbfdb5e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:35.349521 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21829d36-2c9b-4c95-95b5-218fbfdb5e65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:35.355929 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21829d36-2c9b-4c95-95b5-218fbfdb5e65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:35.357462 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21829d36-2c9b-4c95-95b5-218fbfdb5e65 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:35.372445 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-21829d36-2c9b-4c95-95b5-218fbfdb5e65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:49:35.372445 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:35.372445 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:35.372445 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:35.372445 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:35.372445 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:35.372445 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:35.372445 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:35.372445 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:35.372445 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:35.372445 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:35.372445 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:35.372445 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:49:35.372445 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:49:35.372445 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:49:35.372445 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:49:35.373431 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:35.373431 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:35.373431 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:35.373431 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:35.373431 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:35.373431 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:35.373431 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:35.373431 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:35.373431 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:35.373431 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:35.373431 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:35.373431 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:35.373431 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:35.373431 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:35.373431 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:35.373431 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:35.373431 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:49:35.374832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:49:35.374832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:49:35.374832 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:35.374832 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 545/1089] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:49:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:35.380067 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9fcded-a709-474f-b742-4e78aea31e61 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:35.399209 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:35.479434 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9fcded-a709-474f-b742-4e78aea31e61 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:35.513319 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29bf118-6442-482b-8f32-16e02253c522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:35.513921 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29bf118-6442-482b-8f32-16e02253c522 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:35.514203 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29bf118-6442-482b-8f32-16e02253c522 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:35.515103 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29bf118-6442-482b-8f32-16e02253c522 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:35.515883 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29bf118-6442-482b-8f32-16e02253c522 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:35.516188 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29bf118-6442-482b-8f32-16e02253c522 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:35.517592 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29bf118-6442-482b-8f32-16e02253c522 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:35.528432 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 545/1090] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:35.534183 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21829d36-2c9b-4c95-95b5-218fbfdb5e65 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:35.632018 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21829d36-2c9b-4c95-95b5-218fbfdb5e65 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:35.664903 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c135e9-3d07-429f-97c2-404ea544b273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:35.665518 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c135e9-3d07-429f-97c2-404ea544b273 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:35.665796 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c135e9-3d07-429f-97c2-404ea544b273 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:35.666068 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c135e9-3d07-429f-97c2-404ea544b273 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:35.673422 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c135e9-3d07-429f-97c2-404ea544b273 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:35.674846 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c135e9-3d07-429f-97c2-404ea544b273 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:35.685581 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-e9c135e9-3d07-429f-97c2-404ea544b273 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 04 00:49:35.685581 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:35.685581 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:35.685581 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:35.685581 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:35.685581 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:35.685581 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:35.685581 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:35.685581 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:35.685581 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:35.685581 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:35.685581 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:35.685581 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:49:35.685581 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:49:35.685581 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:49:35.685581 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:49:35.687408 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:35.687408 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:35.687408 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:49:35.687408 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:49:35.687408 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:49:35.687408 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:49:35.687408 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:35.687408 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:35.687408 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:35.687408 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:35.687408 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:35.687408 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:35.687408 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:35.687408 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:35.687408 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:35.687408 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:35.687408 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:35.687408 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:35.688608 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:35.688608 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:35.688608 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:49:35.688608 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:49:35.688608 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:49:35.688608 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:49:35.688608 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:49:35.688608 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:49:35.688608 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 04 00:49:35.688608 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:35.688608 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 546/1091] 10.222.0.27 () {56 vars in 1093 bytes} [Sun Sep 4 00:49:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:49:35.691741 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d371902-7815-4a42-95a0-7b321ba7841f None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:35.692176 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d371902-7815-4a42-95a0-7b321ba7841f None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:35.692289 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d371902-7815-4a42-95a0-7b321ba7841f None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:35.693736 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 546/1092] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:49:35.695094 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c135e9-3d07-429f-97c2-404ea544b273 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:35.698452 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c29ac-2dfb-4e65-ac00-066bcf51c1e2 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:35.698737 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c29ac-2dfb-4e65-ac00-066bcf51c1e2 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:35.699008 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c29ac-2dfb-4e65-ac00-066bcf51c1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:35.717321 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-081c29ac-2dfb-4e65-ac00-066bcf51c1e2 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:35.718537 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:35.718834 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-081c29ac-2dfb-4e65-ac00-066bcf51c1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:35.785901 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 547/1093] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 00:49:35.804140 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c135e9-3d07-429f-97c2-404ea544b273 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:35.838086 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e38d4-0397-4f88-83fd-0108c6788b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:35.838679 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e38d4-0397-4f88-83fd-0108c6788b69 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:35.838956 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e38d4-0397-4f88-83fd-0108c6788b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:35.839226 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e38d4-0397-4f88-83fd-0108c6788b69 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:35.839934 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920e38d4-0397-4f88-83fd-0108c6788b69 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:35.840233 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920e38d4-0397-4f88-83fd-0108c6788b69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:35.841672 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920e38d4-0397-4f88-83fd-0108c6788b69 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:35.857946 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 547/1094] 10.222.0.27 () {56 vars in 1133 bytes} [Sun Sep 4 00:49:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:35.864220 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c29ac-2dfb-4e65-ac00-066bcf51c1e2 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:35.872545 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:35.963217 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c29ac-2dfb-4e65-ac00-066bcf51c1e2 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:35.997020 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a0776-1543-4802-9fb8-37d1806b7f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:35.997623 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a0776-1543-4802-9fb8-37d1806b7f81 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:35.997907 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a0776-1543-4802-9fb8-37d1806b7f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:35.998185 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a0776-1543-4802-9fb8-37d1806b7f81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:36.063901 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e38d4-0397-4f88-83fd-0108c6788b69 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:36.065074 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2a0776-1543-4802-9fb8-37d1806b7f81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:36.066530 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2a0776-1543-4802-9fb8-37d1806b7f81 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:36.074560 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:36.141659 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:36.163732 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e38d4-0397-4f88-83fd-0108c6788b69 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:36.165723 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 548/1095] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:49:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 04 00:49:36.172131 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a0776-1543-4802-9fb8-37d1806b7f81 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:36.201039 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e3dff-8b1b-448a-9678-e5e078cf0393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:36.201660 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e3dff-8b1b-448a-9678-e5e078cf0393 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:36.202452 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e3dff-8b1b-448a-9678-e5e078cf0393 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:36.202738 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e3dff-8b1b-448a-9678-e5e078cf0393 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:36.212651 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e3dff-8b1b-448a-9678-e5e078cf0393 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:49:36.217951 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-f47e3dff-8b1b-448a-9678-e5e078cf0393 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:49:36.217951 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:36.217951 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:36.217951 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:36.217951 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:36.217951 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:36.217951 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:36.217951 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:36.217951 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:36.217951 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:36.217951 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:36.217951 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:36.217951 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:49:36.217951 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:49:36.217951 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:49:36.217951 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:49:36.219248 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:49:36.219248 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:49:36.219248 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:49:36.219248 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:49:36.219248 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:36.219248 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:36.219248 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:36.219248 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:36.219248 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:36.219248 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:36.219248 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:36.219248 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:36.219248 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:36.219248 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:36.219248 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:36.219248 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:36.220306 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:36.220306 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:36.220306 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:36.220306 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:36.220306 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:49:36.220306 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:49:36.220306 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:49:36.220306 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:49:36.220306 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:49:36.220306 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:49:36.220306 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:49:36.220306 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:36.221338 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 548/1096] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:49:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:36.227170 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e3dff-8b1b-448a-9678-e5e078cf0393 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:36.278195 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a0776-1543-4802-9fb8-37d1806b7f81 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:36.285934 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:36.313778 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4128967b-6bee-48d8-bbc2-b68a7884623a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:36.314462 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128967b-6bee-48d8-bbc2-b68a7884623a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:36.314681 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128967b-6bee-48d8-bbc2-b68a7884623a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:36.314950 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128967b-6bee-48d8-bbc2-b68a7884623a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:36.325453 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4128967b-6bee-48d8-bbc2-b68a7884623a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:36.326191 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e3dff-8b1b-448a-9678-e5e078cf0393 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:36.326941 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4128967b-6bee-48d8-bbc2-b68a7884623a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:36.341393 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-4128967b-6bee-48d8-bbc2-b68a7884623a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:49:36.341393 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:36.341393 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:36.341393 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:36.341393 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:36.341393 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:36.341393 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:36.341393 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:36.341393 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:36.341393 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:36.341393 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:36.341393 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:36.341393 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:49:36.341393 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:49:36.341393 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:49:36.341393 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:49:36.342892 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:36.342892 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:36.342892 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:36.342892 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:36.342892 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:36.342892 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:36.342892 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:36.342892 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:36.342892 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:36.342892 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:36.342892 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:36.342892 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:36.342892 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:36.342892 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:36.342892 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:36.342892 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:36.342892 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:49:36.344301 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:49:36.344301 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:49:36.344301 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:36.344301 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 549/1097] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:49:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 04 00:49:36.348882 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4128967b-6bee-48d8-bbc2-b68a7884623a None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:36.363067 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee333221-42b4-40f2-9e30-172048c95ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:36.363672 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee333221-42b4-40f2-9e30-172048c95ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:36.363955 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee333221-42b4-40f2-9e30-172048c95ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:36.364226 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee333221-42b4-40f2-9e30-172048c95ba7 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:36.364917 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee333221-42b4-40f2-9e30-172048c95ba7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:36.365221 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee333221-42b4-40f2-9e30-172048c95ba7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:36.366562 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee333221-42b4-40f2-9e30-172048c95ba7 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:36.371537 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:36.375058 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 549/1098] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:49:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:49:36.380778 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee333221-42b4-40f2-9e30-172048c95ba7 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:36.448982 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4128967b-6bee-48d8-bbc2-b68a7884623a None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:36.479775 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee333221-42b4-40f2-9e30-172048c95ba7 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:36.483301 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6833e32-62fc-4adb-b6b7-83cfa8cd2216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:36.483897 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6833e32-62fc-4adb-b6b7-83cfa8cd2216 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:36.484178 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6833e32-62fc-4adb-b6b7-83cfa8cd2216 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:36.484451 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6833e32-62fc-4adb-b6b7-83cfa8cd2216 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:36.485106 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6833e32-62fc-4adb-b6b7-83cfa8cd2216 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:36.485406 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6833e32-62fc-4adb-b6b7-83cfa8cd2216 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:36.486775 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6833e32-62fc-4adb-b6b7-83cfa8cd2216 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:36.498477 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 550/1099] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:49:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:49:36.504118 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6833e32-62fc-4adb-b6b7-83cfa8cd2216 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:36.512559 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0946c39-2106-437d-9a00-cddcdc3e8207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:36.513181 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0946c39-2106-437d-9a00-cddcdc3e8207 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:36.513459 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0946c39-2106-437d-9a00-cddcdc3e8207 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:36.513725 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0946c39-2106-437d-9a00-cddcdc3e8207 None admin] PATH_INFO: `/v3/users/42c7e11f1b254c58aa492374034f0900` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:36.515336 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0946c39-2106-437d-9a00-cddcdc3e8207 None admin] RBAC: Authorizing `identity:get_user(user_id=42c7e11f1b254c58aa492374034f0900)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:36.516775 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0946c39-2106-437d-9a00-cddcdc3e8207 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:36.518599 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 550/1100] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:49:36 2022] GET /identity/v3/users/42c7e11f1b254c58aa492374034f0900 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:49:36.522510 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:36.524068 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0946c39-2106-437d-9a00-cddcdc3e8207 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:36.590454 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:36.603000 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6833e32-62fc-4adb-b6b7-83cfa8cd2216 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:36.625276 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0946c39-2106-437d-9a00-cddcdc3e8207 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:36.636172 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80358ef4-ca30-4b36-8a06-4315c4daac2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:36.636824 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80358ef4-ca30-4b36-8a06-4315c4daac2a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:36.637104 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80358ef4-ca30-4b36-8a06-4315c4daac2a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:36.637378 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80358ef4-ca30-4b36-8a06-4315c4daac2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:36.644398 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80358ef4-ca30-4b36-8a06-4315c4daac2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:36.645862 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80358ef4-ca30-4b36-8a06-4315c4daac2a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:36.657636 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-80358ef4-ca30-4b36-8a06-4315c4daac2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:49:36.657636 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:36.657636 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:36.657636 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:36.657636 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:36.657636 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:36.657636 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:36.657636 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:36.657636 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:36.657636 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:36.657636 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:36.657636 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:36.657636 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:49:36.657636 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:49:36.657636 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:49:36.658640 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:49:36.658640 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:36.658640 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:36.658640 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:36.658640 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:36.658640 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:36.658640 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:36.658640 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:36.658640 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:36.658640 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:36.658640 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:36.658640 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:36.658640 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:36.658640 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:36.658640 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:36.658640 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:36.658640 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:36.660060 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:49:36.660060 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:49:36.660060 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:49:36.660060 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:49:36.660060 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:49:36.660060 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:49:36.660060 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:49:36.660060 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:49:36.660060 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 551/1101] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:49:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:36.664243 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ded6d3-0f0a-4931-8abc-9e52f1c6e19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:36.664662 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80358ef4-ca30-4b36-8a06-4315c4daac2a None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:36.664887 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ded6d3-0f0a-4931-8abc-9e52f1c6e19b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:36.665272 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ded6d3-0f0a-4931-8abc-9e52f1c6e19b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:36.665549 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ded6d3-0f0a-4931-8abc-9e52f1c6e19b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:36.733668 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ded6d3-0f0a-4931-8abc-9e52f1c6e19b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:36.735130 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ded6d3-0f0a-4931-8abc-9e52f1c6e19b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:36.741562 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:36.761681 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80358ef4-ca30-4b36-8a06-4315c4daac2a None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:36.795949 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e05b04a-469a-43e4-b9de-ad89f290102e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:36.796573 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e05b04a-469a-43e4-b9de-ad89f290102e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:36.796854 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e05b04a-469a-43e4-b9de-ad89f290102e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:36.797128 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e05b04a-469a-43e4-b9de-ad89f290102e None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:36.797811 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e05b04a-469a-43e4-b9de-ad89f290102e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:36.798115 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e05b04a-469a-43e4-b9de-ad89f290102e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:36.799539 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e05b04a-469a-43e4-b9de-ad89f290102e None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:36.799931 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:36.811248 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 552/1102] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:49:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:49:36.817442 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e05b04a-469a-43e4-b9de-ad89f290102e None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:36.834785 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 551/1103] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:49:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 04 00:49:36.840608 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ded6d3-0f0a-4931-8abc-9e52f1c6e19b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:36.917304 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e05b04a-469a-43e4-b9de-ad89f290102e None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:36.940215 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ded6d3-0f0a-4931-8abc-9e52f1c6e19b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:36.952363 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b18af7f-d4a6-476c-b33a-2e3043315801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:36.953094 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18af7f-d4a6-476c-b33a-2e3043315801 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:36.953372 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18af7f-d4a6-476c-b33a-2e3043315801 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:36.953681 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18af7f-d4a6-476c-b33a-2e3043315801 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:36.954343 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b18af7f-d4a6-476c-b33a-2e3043315801 None admin] RBAC: Adding query filter params (role.id=91d340dbd1f142c5aa33ad640eca891c, scope.project.id=5100a8d316d243fb8f82e9044e1b3b06, user.id=68bc00bda5194cd8a7928f8c4df4ccd1) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:36.954647 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b18af7f-d4a6-476c-b33a-2e3043315801 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:36.956061 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b18af7f-d4a6-476c-b33a-2e3043315801 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:36.961709 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 553/1104] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:49:36 2022] GET /identity/v3/role_assignments?user.id=68bc00bda5194cd8a7928f8c4df4ccd1&scope.project.id=5100a8d316d243fb8f82e9044e1b3b06&role.id=91d340dbd1f142c5aa33ad640eca891c => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:49:36.975216 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f3d04-c5c0-48d2-b7db-b023a475e50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:36.975869 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f3d04-c5c0-48d2-b7db-b023a475e50b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:36.976149 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f3d04-c5c0-48d2-b7db-b023a475e50b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:36.976422 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f3d04-c5c0-48d2-b7db-b023a475e50b None admin] PATH_INFO: `/v3/projects/fd852c22be454accb5dd60e3b6e60ef7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:36.977657 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f3d04-c5c0-48d2-b7db-b023a475e50b None admin] RBAC: Authorizing `identity:get_project(project_id=fd852c22be454accb5dd60e3b6e60ef7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:36.979140 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f3d04-c5c0-48d2-b7db-b023a475e50b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:36.980928 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 552/1105] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:49:36 2022] GET /identity/v3/projects/fd852c22be454accb5dd60e3b6e60ef7 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:49:36.987486 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b18af7f-d4a6-476c-b33a-2e3043315801 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:36.990470 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:37.094650 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b18af7f-d4a6-476c-b33a-2e3043315801 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:37.130492 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a5b35-16e9-4a46-8b77-b72b92936305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:37.131097 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a5b35-16e9-4a46-8b77-b72b92936305 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:37.131374 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a5b35-16e9-4a46-8b77-b72b92936305 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:37.131643 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a5b35-16e9-4a46-8b77-b72b92936305 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:37.132357 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045a5b35-16e9-4a46-8b77-b72b92936305 None admin] RBAC: Adding query filter params (role.id=c8a59d34ee2f4cefa6940cac778260c7, scope.project.id=fd852c22be454accb5dd60e3b6e60ef7, user.id=42c7e11f1b254c58aa492374034f0900) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:37.132694 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045a5b35-16e9-4a46-8b77-b72b92936305 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:37.134114 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045a5b35-16e9-4a46-8b77-b72b92936305 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:37.139814 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 554/1106] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:49:36 2022] GET /identity/v3/role_assignments?user.id=42c7e11f1b254c58aa492374034f0900&scope.project.id=fd852c22be454accb5dd60e3b6e60ef7&role.id=c8a59d34ee2f4cefa6940cac778260c7 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:49:39.098256 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd0bed-4984-4526-925e-8125201ebd50 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:39.098784 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd0bed-4984-4526-925e-8125201ebd50 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:39.098784 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd0bed-4984-4526-925e-8125201ebd50 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:39.100138 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 553/1107] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:49:39.105249 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41b9d7-7a1c-4394-b445-c25d6869ead4 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:39.105617 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41b9d7-7a1c-4394-b445-c25d6869ead4 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:39.105799 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41b9d7-7a1c-4394-b445-c25d6869ead4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:39.125187 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-6b41b9d7-7a1c-4394-b445-c25d6869ead4 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:39.126363 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-6b41b9d7-7a1c-4394-b445-c25d6869ead4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:39.184748 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:39.192165 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 555/1108] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:49:39.462385 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd0bed-4984-4526-925e-8125201ebd50 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:39.498028 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:39.563782 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd0bed-4984-4526-925e-8125201ebd50 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:39.597164 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f632e-d7ed-46a0-80d5-bd0231e9f5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:39.597775 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f632e-d7ed-46a0-80d5-bd0231e9f5c6 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:39.598057 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f632e-d7ed-46a0-80d5-bd0231e9f5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:39.598332 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f632e-d7ed-46a0-80d5-bd0231e9f5c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:39.599427 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f632e-d7ed-46a0-80d5-bd0231e9f5c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:39.600872 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f632e-d7ed-46a0-80d5-bd0231e9f5c6 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:39.602424 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 554/1109] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:49:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:39.608093 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b41b9d7-7a1c-4394-b445-c25d6869ead4 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:39.703866 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b41b9d7-7a1c-4394-b445-c25d6869ead4 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:39.736538 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf5eea-9229-4fb0-ab14-c4aa215b2b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:39.737071 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf5eea-9229-4fb0-ab14-c4aa215b2b77 None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:39.737349 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf5eea-9229-4fb0-ab14-c4aa215b2b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:39.737618 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf5eea-9229-4fb0-ab14-c4aa215b2b77 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:39.738266 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf5eea-9229-4fb0-ab14-c4aa215b2b77 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:39.739684 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf5eea-9229-4fb0-ab14-c4aa215b2b77 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:39.829491 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 556/1110] 10.222.0.27 () {60 vars in 1132 bytes} [Sun Sep 4 00:49:39 2022] POST /identity/v3/users => generated 286 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:49:41.783936 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a494be-3eaa-4a85-8960-5a9e3150caf2 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:41.784485 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a494be-3eaa-4a85-8960-5a9e3150caf2 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:41.784606 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a494be-3eaa-4a85-8960-5a9e3150caf2 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:41.785843 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 555/1111] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:49:41.791251 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b4ba9-0f78-4bb2-acb0-aa36a0d35552 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:41.791625 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b4ba9-0f78-4bb2-acb0-aa36a0d35552 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:41.791807 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b4ba9-0f78-4bb2-acb0-aa36a0d35552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:41.809718 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-cd1b4ba9-0f78-4bb2-acb0-aa36a0d35552 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:41.810927 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-cd1b4ba9-0f78-4bb2-acb0-aa36a0d35552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:41.856425 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:41.875474 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 557/1112] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:49:42.155318 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a494be-3eaa-4a85-8960-5a9e3150caf2 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:42.257038 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a494be-3eaa-4a85-8960-5a9e3150caf2 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:42.259274 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:42.291799 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158f74a-d8de-4793-b3f6-9c19fde8fbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:42.292420 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158f74a-d8de-4793-b3f6-9c19fde8fbb5 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:42.292730 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158f74a-d8de-4793-b3f6-9c19fde8fbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:42.293003 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158f74a-d8de-4793-b3f6-9c19fde8fbb5 None admin] PATH_INFO: `/v3/roles/7df9ebc56a2a4214b2b64a5941812ef0` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:42.294374 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158f74a-d8de-4793-b3f6-9c19fde8fbb5 None admin] RBAC: Authorizing `identity:get_role(role_id=7df9ebc56a2a4214b2b64a5941812ef0)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:42.296237 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158f74a-d8de-4793-b3f6-9c19fde8fbb5 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:42.297590 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 556/1113] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:49:42 2022] GET /identity/v3/roles/7df9ebc56a2a4214b2b64a5941812ef0 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:42.303407 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b4ba9-0f78-4bb2-acb0-aa36a0d35552 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:42.402471 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b4ba9-0f78-4bb2-acb0-aa36a0d35552 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:42.435037 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c99516-199c-40b6-85d2-4224927ccde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:42.435597 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c99516-199c-40b6-85d2-4224927ccde6 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:42.436397 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c99516-199c-40b6-85d2-4224927ccde6 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:42.436784 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c99516-199c-40b6-85d2-4224927ccde6 None admin] PATH_INFO: `/v3/users/c8c1b5cc8f6f4264b834b1083c11a6a5` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:42.449156 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c99516-199c-40b6-85d2-4224927ccde6 None admin] RBAC: Authorizing `identity:get_user(user_id=c8c1b5cc8f6f4264b834b1083c11a6a5)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:42.450584 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c99516-199c-40b6-85d2-4224927ccde6 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:42.452700 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 558/1114] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:49:42 2022] GET /identity/v3/users/c8c1b5cc8f6f4264b834b1083c11a6a5 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:42.458486 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158f74a-d8de-4793-b3f6-9c19fde8fbb5 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:42.556830 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158f74a-d8de-4793-b3f6-9c19fde8fbb5 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:42.590360 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903344a-9d60-4ef6-92f7-30b98001ef37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:42.590945 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903344a-9d60-4ef6-92f7-30b98001ef37 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:42.591226 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903344a-9d60-4ef6-92f7-30b98001ef37 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:42.591498 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903344a-9d60-4ef6-92f7-30b98001ef37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:42.631827 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:42.657904 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e903344a-9d60-4ef6-92f7-30b98001ef37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:42.659339 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e903344a-9d60-4ef6-92f7-30b98001ef37 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:42.756159 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 557/1115] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:49:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 04 00:49:42.762102 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c99516-199c-40b6-85d2-4224927ccde6 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:42.802731 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:42.859124 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c99516-199c-40b6-85d2-4224927ccde6 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:42.891220 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc5ed3-9d65-4f4c-93b2-eac0cfc00cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:42.891828 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc5ed3-9d65-4f4c-93b2-eac0cfc00cf2 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:42.892106 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc5ed3-9d65-4f4c-93b2-eac0cfc00cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:42.892375 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc5ed3-9d65-4f4c-93b2-eac0cfc00cf2 None admin] PATH_INFO: `/v3/projects/fd852c22be454accb5dd60e3b6e60ef7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:42.893490 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc5ed3-9d65-4f4c-93b2-eac0cfc00cf2 None admin] RBAC: Authorizing `identity:get_project(project_id=fd852c22be454accb5dd60e3b6e60ef7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:42.894911 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc5ed3-9d65-4f4c-93b2-eac0cfc00cf2 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:42.896588 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 559/1116] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:49:42 2022] GET /identity/v3/projects/fd852c22be454accb5dd60e3b6e60ef7 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:42.902719 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903344a-9d60-4ef6-92f7-30b98001ef37 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:42.971268 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:42.999565 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903344a-9d60-4ef6-92f7-30b98001ef37 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:43.032392 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bafba7c-6c16-4d44-ae53-ed6a2002ecc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:43.032945 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bafba7c-6c16-4d44-ae53-ed6a2002ecc3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:43.033223 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bafba7c-6c16-4d44-ae53-ed6a2002ecc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:43.033493 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bafba7c-6c16-4d44-ae53-ed6a2002ecc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:43.034192 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bafba7c-6c16-4d44-ae53-ed6a2002ecc3 None admin] RBAC: Adding query filter params (role.id=7df9ebc56a2a4214b2b64a5941812ef0, scope.project.id=fd852c22be454accb5dd60e3b6e60ef7, user.id=c8c1b5cc8f6f4264b834b1083c11a6a5) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:43.034495 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bafba7c-6c16-4d44-ae53-ed6a2002ecc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:43.035898 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bafba7c-6c16-4d44-ae53-ed6a2002ecc3 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:43.041343 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 558/1117] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:49:42 2022] GET /identity/v3/role_assignments?user.id=c8c1b5cc8f6f4264b834b1083c11a6a5&scope.project.id=fd852c22be454accb5dd60e3b6e60ef7&role.id=7df9ebc56a2a4214b2b64a5941812ef0 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:44.918397 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168c6cb-1292-4e8f-a7c4-b61ef0c9dee3 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:44.919037 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168c6cb-1292-4e8f-a7c4-b61ef0c9dee3 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:44.919037 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168c6cb-1292-4e8f-a7c4-b61ef0c9dee3 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:44.920843 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 560/1118] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:49:44.926003 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1f273-d0e3-42f5-afa2-17806834ad54 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:44.926420 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1f273-d0e3-42f5-afa2-17806834ad54 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:44.926554 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1f273-d0e3-42f5-afa2-17806834ad54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:44.944914 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f3e1f273-d0e3-42f5-afa2-17806834ad54 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:44.946095 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f3e1f273-d0e3-42f5-afa2-17806834ad54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:45.009983 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 559/1119] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:49:45.283038 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168c6cb-1292-4e8f-a7c4-b61ef0c9dee3 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:45.382532 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168c6cb-1292-4e8f-a7c4-b61ef0c9dee3 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:45.392725 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:45.416186 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68848b-37aa-4d39-989e-71398afcaa1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:45.416829 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68848b-37aa-4d39-989e-71398afcaa1f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:45.417113 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68848b-37aa-4d39-989e-71398afcaa1f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:45.417387 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68848b-37aa-4d39-989e-71398afcaa1f None admin] PATH_INFO: `/v3/roles/7df9ebc56a2a4214b2b64a5941812ef0` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:45.418765 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b68848b-37aa-4d39-989e-71398afcaa1f None admin] RBAC: Authorizing `identity:get_role(role_id=7df9ebc56a2a4214b2b64a5941812ef0)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:45.420168 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b68848b-37aa-4d39-989e-71398afcaa1f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:45.421585 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 561/1120] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:49:45 2022] GET /identity/v3/roles/7df9ebc56a2a4214b2b64a5941812ef0 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:45.427373 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1f273-d0e3-42f5-afa2-17806834ad54 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:45.491505 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:45.525716 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1f273-d0e3-42f5-afa2-17806834ad54 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:45.558414 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e86523a-15ac-4229-bc8e-0cfa76ec2f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:45.559030 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86523a-15ac-4229-bc8e-0cfa76ec2f0e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:45.559311 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86523a-15ac-4229-bc8e-0cfa76ec2f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:45.559582 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86523a-15ac-4229-bc8e-0cfa76ec2f0e None admin] PATH_INFO: `/v3/users/c8c1b5cc8f6f4264b834b1083c11a6a5` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:45.561291 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e86523a-15ac-4229-bc8e-0cfa76ec2f0e None admin] RBAC: Authorizing `identity:get_user(user_id=c8c1b5cc8f6f4264b834b1083c11a6a5)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:45.562710 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e86523a-15ac-4229-bc8e-0cfa76ec2f0e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:45.564273 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 560/1121] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:49:45 2022] GET /identity/v3/users/c8c1b5cc8f6f4264b834b1083c11a6a5 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:45.570052 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68848b-37aa-4d39-989e-71398afcaa1f None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:45.666173 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68848b-37aa-4d39-989e-71398afcaa1f None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:45.699921 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316eaf0c-1d09-44d1-aeec-75c6ddb3a0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:45.700561 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316eaf0c-1d09-44d1-aeec-75c6ddb3a0ed None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:45.700844 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316eaf0c-1d09-44d1-aeec-75c6ddb3a0ed None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:45.701119 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316eaf0c-1d09-44d1-aeec-75c6ddb3a0ed None admin] PATH_INFO: `/v3/projects/fd852c22be454accb5dd60e3b6e60ef7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:45.702447 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316eaf0c-1d09-44d1-aeec-75c6ddb3a0ed None admin] RBAC: Authorizing `identity:get_project(project_id=fd852c22be454accb5dd60e3b6e60ef7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:45.703905 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316eaf0c-1d09-44d1-aeec-75c6ddb3a0ed None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:45.705611 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 562/1122] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:49:45 2022] GET /identity/v3/projects/fd852c22be454accb5dd60e3b6e60ef7 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:45.711436 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e86523a-15ac-4229-bc8e-0cfa76ec2f0e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:45.808544 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e86523a-15ac-4229-bc8e-0cfa76ec2f0e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:45.840915 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95df496-44ea-4592-8a41-b93f826c6d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:45.841553 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95df496-44ea-4592-8a41-b93f826c6d4c None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:45.841838 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95df496-44ea-4592-8a41-b93f826c6d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:45.842723 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95df496-44ea-4592-8a41-b93f826c6d4c None admin] PATH_INFO: `/v3/projects/fd852c22be454accb5dd60e3b6e60ef7/users/c8c1b5cc8f6f4264b834b1083c11a6a5/roles/7df9ebc56a2a4214b2b64a5941812ef0` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:45.844862 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95df496-44ea-4592-8a41-b93f826c6d4c None admin] RBAC: Authorizing `identity:create_grant(project_id=fd852c22be454accb5dd60e3b6e60ef7, user_id=c8c1b5cc8f6f4264b834b1083c11a6a5, role_id=7df9ebc56a2a4214b2b64a5941812ef0)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:45.846273 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95df496-44ea-4592-8a41-b93f826c6d4c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:45.848510 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:45.927996 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 561/1123] 10.222.0.27 () {58 vars in 1439 bytes} [Sun Sep 4 00:49:45 2022] PUT /identity/v3/projects/fd852c22be454accb5dd60e3b6e60ef7/users/c8c1b5cc8f6f4264b834b1083c11a6a5/roles/7df9ebc56a2a4214b2b64a5941812ef0 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 00:49:47.801927 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0775a-9c0a-4414-87fe-8e4240c1b6bf None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:47.802475 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0775a-9c0a-4414-87fe-8e4240c1b6bf None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:47.802475 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0775a-9c0a-4414-87fe-8e4240c1b6bf None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:47.803797 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 563/1124] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:49:47.808875 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76443f28-ffc1-4530-a53f-24ee4d7a4892 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:47.809366 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76443f28-ffc1-4530-a53f-24ee4d7a4892 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:47.809474 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76443f28-ffc1-4530-a53f-24ee4d7a4892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:47.827867 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-76443f28-ffc1-4530-a53f-24ee4d7a4892 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:47.829092 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-76443f28-ffc1-4530-a53f-24ee4d7a4892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:47.894316 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 562/1125] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:49:48.161252 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c0775a-9c0a-4414-87fe-8e4240c1b6bf None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:48.207750 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:48.261385 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c0775a-9c0a-4414-87fe-8e4240c1b6bf None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:48.294646 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11004df1-47aa-4377-95c6-6eb73d5d1589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:48.296041 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11004df1-47aa-4377-95c6-6eb73d5d1589 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:48.296322 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11004df1-47aa-4377-95c6-6eb73d5d1589 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:48.296624 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11004df1-47aa-4377-95c6-6eb73d5d1589 None admin] PATH_INFO: `/v3/roles/7df9ebc56a2a4214b2b64a5941812ef0` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:48.297990 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11004df1-47aa-4377-95c6-6eb73d5d1589 None admin] RBAC: Authorizing `identity:get_role(role_id=7df9ebc56a2a4214b2b64a5941812ef0)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:48.299356 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11004df1-47aa-4377-95c6-6eb73d5d1589 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:48.300672 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 564/1126] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:49:48 2022] GET /identity/v3/roles/7df9ebc56a2a4214b2b64a5941812ef0 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:48.306519 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76443f28-ffc1-4530-a53f-24ee4d7a4892 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:48.400245 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76443f28-ffc1-4530-a53f-24ee4d7a4892 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:48.424434 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:48.434496 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4410227-dc1a-44b4-b26a-5a9f274af68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:48.435806 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4410227-dc1a-44b4-b26a-5a9f274af68a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:48.435806 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4410227-dc1a-44b4-b26a-5a9f274af68a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:48.435806 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4410227-dc1a-44b4-b26a-5a9f274af68a None admin] PATH_INFO: `/v3/users/c8c1b5cc8f6f4264b834b1083c11a6a5` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:48.436971 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4410227-dc1a-44b4-b26a-5a9f274af68a None admin] RBAC: Authorizing `identity:get_user(user_id=c8c1b5cc8f6f4264b834b1083c11a6a5)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:48.438377 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4410227-dc1a-44b4-b26a-5a9f274af68a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:48.439924 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 563/1127] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:49:48 2022] GET /identity/v3/users/c8c1b5cc8f6f4264b834b1083c11a6a5 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:48.445711 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11004df1-47aa-4377-95c6-6eb73d5d1589 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:48.543489 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11004df1-47aa-4377-95c6-6eb73d5d1589 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:48.556116 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:48.576020 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6efe986-d898-48f8-820c-aa628bbc452f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:48.577202 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6efe986-d898-48f8-820c-aa628bbc452f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:48.577534 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6efe986-d898-48f8-820c-aa628bbc452f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:48.577808 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6efe986-d898-48f8-820c-aa628bbc452f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:48.642577 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6efe986-d898-48f8-820c-aa628bbc452f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:48.644005 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6efe986-d898-48f8-820c-aa628bbc452f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:48.737691 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 565/1128] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:49:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 04 00:49:48.743448 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4410227-dc1a-44b4-b26a-5a9f274af68a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:48.839194 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4410227-dc1a-44b4-b26a-5a9f274af68a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:48.871779 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59786897-bcec-4231-98d5-8ad42853115d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:48.872382 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59786897-bcec-4231-98d5-8ad42853115d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:48.872690 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59786897-bcec-4231-98d5-8ad42853115d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:48.873472 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59786897-bcec-4231-98d5-8ad42853115d None admin] PATH_INFO: `/v3/projects/fd852c22be454accb5dd60e3b6e60ef7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:48.874588 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59786897-bcec-4231-98d5-8ad42853115d None admin] RBAC: Authorizing `identity:get_project(project_id=fd852c22be454accb5dd60e3b6e60ef7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:48.875987 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59786897-bcec-4231-98d5-8ad42853115d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:48.877610 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 564/1129] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:49:48 2022] GET /identity/v3/projects/fd852c22be454accb5dd60e3b6e60ef7 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:48.883752 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6efe986-d898-48f8-820c-aa628bbc452f None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:48.898054 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:48.979363 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6efe986-d898-48f8-820c-aa628bbc452f None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:49.012277 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d88497b-b63c-4fd4-9186-4753f7f6bb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:49.012885 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88497b-b63c-4fd4-9186-4753f7f6bb99 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:49.013164 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88497b-b63c-4fd4-9186-4753f7f6bb99 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:49.013435 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88497b-b63c-4fd4-9186-4753f7f6bb99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:49.014723 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d88497b-b63c-4fd4-9186-4753f7f6bb99 None admin] RBAC: Adding query filter params (role.id=7df9ebc56a2a4214b2b64a5941812ef0, scope.project.id=fd852c22be454accb5dd60e3b6e60ef7, user.id=c8c1b5cc8f6f4264b834b1083c11a6a5) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:49.015022 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d88497b-b63c-4fd4-9186-4753f7f6bb99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:49.016440 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d88497b-b63c-4fd4-9186-4753f7f6bb99 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:49.021985 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 566/1130] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:49:48 2022] GET /identity/v3/role_assignments?user.id=c8c1b5cc8f6f4264b834b1083c11a6a5&scope.project.id=fd852c22be454accb5dd60e3b6e60ef7&role.id=7df9ebc56a2a4214b2b64a5941812ef0 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:50.920548 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4660be8-fcdc-4c55-ae19-26ae68f29751 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:50.921097 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4660be8-fcdc-4c55-ae19-26ae68f29751 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:50.921097 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4660be8-fcdc-4c55-ae19-26ae68f29751 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:50.922466 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 565/1131] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:49:50.927764 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca1a92-2990-4282-95e5-dfed02f26736 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:50.928200 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca1a92-2990-4282-95e5-dfed02f26736 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:50.928295 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca1a92-2990-4282-95e5-dfed02f26736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:50.946777 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-40ca1a92-2990-4282-95e5-dfed02f26736 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:50.947969 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-40ca1a92-2990-4282-95e5-dfed02f26736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:51.012364 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 567/1132] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:49:51.282182 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4660be8-fcdc-4c55-ae19-26ae68f29751 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:51.341546 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:51.381649 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4660be8-fcdc-4c55-ae19-26ae68f29751 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:51.415222 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f24d0-881f-4617-8199-e22da6660676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:51.415834 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f24d0-881f-4617-8199-e22da6660676 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:51.416116 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f24d0-881f-4617-8199-e22da6660676 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:51.416389 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f24d0-881f-4617-8199-e22da6660676 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:51.417515 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f24d0-881f-4617-8199-e22da6660676 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:51.418928 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f24d0-881f-4617-8199-e22da6660676 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:51.420744 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 566/1133] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:49:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:51.426556 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca1a92-2990-4282-95e5-dfed02f26736 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:51.434493 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:51.521676 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca1a92-2990-4282-95e5-dfed02f26736 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:51.554230 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe9395-9a80-42b2-81b3-a3107e1cd163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:51.554817 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe9395-9a80-42b2-81b3-a3107e1cd163 None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:51.555096 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe9395-9a80-42b2-81b3-a3107e1cd163 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:51.555369 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe9395-9a80-42b2-81b3-a3107e1cd163 None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:51.556004 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe9395-9a80-42b2-81b3-a3107e1cd163 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:51.557459 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe9395-9a80-42b2-81b3-a3107e1cd163 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:51.666650 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 568/1134] 10.222.0.27 () {60 vars in 1141 bytes} [Sun Sep 4 00:49:51 2022] POST /identity/v3/projects => generated 311 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:49:53.587627 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe46cf-344f-4114-9df4-f542d2d118f0 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:53.588167 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe46cf-344f-4114-9df4-f542d2d118f0 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:53.588167 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe46cf-344f-4114-9df4-f542d2d118f0 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:53.589558 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 567/1135] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:49:53.595158 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6506665-b5da-490f-bb29-dece20b53c6e None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:53.595532 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6506665-b5da-490f-bb29-dece20b53c6e None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:53.595719 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6506665-b5da-490f-bb29-dece20b53c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:53.614094 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-c6506665-b5da-490f-bb29-dece20b53c6e None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:53.615285 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c6506665-b5da-490f-bb29-dece20b53c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:53.667337 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:53.680820 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 569/1136] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:49:53.947051 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe46cf-344f-4114-9df4-f542d2d118f0 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:54.046659 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe46cf-344f-4114-9df4-f542d2d118f0 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:54.074441 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:54.082403 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b45e2-1e19-4738-bae2-48230e8157fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:54.083029 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b45e2-1e19-4738-bae2-48230e8157fd None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:54.090188 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b45e2-1e19-4738-bae2-48230e8157fd None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:54.090188 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b45e2-1e19-4738-bae2-48230e8157fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:54.090188 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942b45e2-1e19-4738-bae2-48230e8157fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:54.090188 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942b45e2-1e19-4738-bae2-48230e8157fd None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:54.090188 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 568/1137] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:49:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:54.094181 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6506665-b5da-490f-bb29-dece20b53c6e None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:54.196810 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6506665-b5da-490f-bb29-dece20b53c6e None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:54.229887 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5925fc8-82b8-4b88-b10d-2a234b5ce3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:54.230498 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5925fc8-82b8-4b88-b10d-2a234b5ce3da None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:54.230780 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5925fc8-82b8-4b88-b10d-2a234b5ce3da None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:54.231056 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5925fc8-82b8-4b88-b10d-2a234b5ce3da None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:54.231706 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5925fc8-82b8-4b88-b10d-2a234b5ce3da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:54.233803 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5925fc8-82b8-4b88-b10d-2a234b5ce3da None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:54.319895 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 570/1138] 10.222.0.27 () {60 vars in 1132 bytes} [Sun Sep 4 00:49:54 2022] POST /identity/v3/users => generated 286 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:49:56.239610 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aed7e0-9eaa-475a-ab1e-b7a8a747d479 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:56.240085 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aed7e0-9eaa-475a-ab1e-b7a8a747d479 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:56.240189 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aed7e0-9eaa-475a-ab1e-b7a8a747d479 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:56.241537 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 569/1139] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:49:56.246626 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf7054-5239-41b3-86dc-453e0839c9d6 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:56.247051 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf7054-5239-41b3-86dc-453e0839c9d6 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:56.247179 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf7054-5239-41b3-86dc-453e0839c9d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:56.266484 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-c0bf7054-5239-41b3-86dc-453e0839c9d6 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:56.267670 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c0bf7054-5239-41b3-86dc-453e0839c9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:56.306908 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:56.332558 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 571/1140] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:49:56.600170 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aed7e0-9eaa-475a-ab1e-b7a8a747d479 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:56.698092 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aed7e0-9eaa-475a-ab1e-b7a8a747d479 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:56.730609 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518bd325-c725-4d16-8e65-fd5eeefad7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:56.731214 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518bd325-c725-4d16-8e65-fd5eeefad7b8 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:56.731494 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518bd325-c725-4d16-8e65-fd5eeefad7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:56.731765 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518bd325-c725-4d16-8e65-fd5eeefad7b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:56.742009 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518bd325-c725-4d16-8e65-fd5eeefad7b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:49:56.746712 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-518bd325-c725-4d16-8e65-fd5eeefad7b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:49:56.746712 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:49:56.746712 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:49:56.746712 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:49:56.746712 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:49:56.746712 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:49:56.746712 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:49:56.746712 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:49:56.746712 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:49:56.746712 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:49:56.746712 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:49:56.746712 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:49:56.746712 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:49:56.746712 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:49:56.746712 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:49:56.746712 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:49:56.748475 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:49:56.748475 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:49:56.748475 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:49:56.748475 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:49:56.748475 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:49:56.748475 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:49:56.748475 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:49:56.748475 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:49:56.748475 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:49:56.748475 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:49:56.748475 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:49:56.748475 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:49:56.748475 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:49:56.748475 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:49:56.748475 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:49:56.748475 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:49:56.749792 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:49:56.749792 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:49:56.749792 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:49:56.749792 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:49:56.749792 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:49:56.749792 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:49:56.749792 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:49:56.749792 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:49:56.749792 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:49:56.749792 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:49:56.749792 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:49:56.749792 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:49:56.749792 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 570/1141] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:49:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:49:56.754313 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf7054-5239-41b3-86dc-453e0839c9d6 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:56.849017 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf7054-5239-41b3-86dc-453e0839c9d6 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:56.882567 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04595474-a2ca-4161-a0e1-3274a9d188d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:56.883096 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04595474-a2ca-4161-a0e1-3274a9d188d5 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:56.883372 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04595474-a2ca-4161-a0e1-3274a9d188d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:56.883642 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04595474-a2ca-4161-a0e1-3274a9d188d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:56.884291 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04595474-a2ca-4161-a0e1-3274a9d188d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:56.884617 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04595474-a2ca-4161-a0e1-3274a9d188d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:56.885974 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04595474-a2ca-4161-a0e1-3274a9d188d5 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:56.893817 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 572/1142] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:49:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:56.900736 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518bd325-c725-4d16-8e65-fd5eeefad7b8 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:56.955603 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:56.998834 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518bd325-c725-4d16-8e65-fd5eeefad7b8 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:57.030909 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2d498-f2ce-4f78-a40c-7a8258e28898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:57.031514 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2d498-f2ce-4f78-a40c-7a8258e28898 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:57.031796 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2d498-f2ce-4f78-a40c-7a8258e28898 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:57.032072 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2d498-f2ce-4f78-a40c-7a8258e28898 None admin] PATH_INFO: `/v3/users/bd3d37df64484ce08cbaf6fccb693efa` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:57.045373 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f2d498-f2ce-4f78-a40c-7a8258e28898 None admin] RBAC: Authorizing `identity:get_user(user_id=bd3d37df64484ce08cbaf6fccb693efa)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:57.046803 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f2d498-f2ce-4f78-a40c-7a8258e28898 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:57.048377 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 571/1143] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:49:56 2022] GET /identity/v3/users/bd3d37df64484ce08cbaf6fccb693efa => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:57.054467 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04595474-a2ca-4161-a0e1-3274a9d188d5 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:57.090809 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:57.158402 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04595474-a2ca-4161-a0e1-3274a9d188d5 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:57.191874 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52080e14-56df-49bb-a712-2459177c1fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:57.192525 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52080e14-56df-49bb-a712-2459177c1fb3 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:57.192778 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52080e14-56df-49bb-a712-2459177c1fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:57.193054 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52080e14-56df-49bb-a712-2459177c1fb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:57.261349 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52080e14-56df-49bb-a712-2459177c1fb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:57.262807 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52080e14-56df-49bb-a712-2459177c1fb3 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:57.297188 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:57.361078 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 573/1144] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:49:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 04 00:49:57.367218 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2d498-f2ce-4f78-a40c-7a8258e28898 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:57.464545 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2d498-f2ce-4f78-a40c-7a8258e28898 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:57.478609 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:57.498881 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ed81f-6669-4822-bd7b-84b6c0cab9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:57.499520 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ed81f-6669-4822-bd7b-84b6c0cab9e9 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:57.499804 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ed81f-6669-4822-bd7b-84b6c0cab9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:57.500080 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ed81f-6669-4822-bd7b-84b6c0cab9e9 None admin] PATH_INFO: `/v3/projects/cd79b6bfb8a449b2a2d0c1f9fe59678f` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:57.501229 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843ed81f-6669-4822-bd7b-84b6c0cab9e9 None admin] RBAC: Authorizing `identity:get_project(project_id=cd79b6bfb8a449b2a2d0c1f9fe59678f)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:57.502656 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843ed81f-6669-4822-bd7b-84b6c0cab9e9 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:57.504284 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 572/1145] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:49:57 2022] GET /identity/v3/projects/cd79b6bfb8a449b2a2d0c1f9fe59678f => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:57.510492 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52080e14-56df-49bb-a712-2459177c1fb3 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:57.605942 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52080e14-56df-49bb-a712-2459177c1fb3 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:49:57.637788 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de009f11-0f83-41eb-88e0-8eabcf7f2b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:49:57.638999 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de009f11-0f83-41eb-88e0-8eabcf7f2b08 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:57.639287 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de009f11-0f83-41eb-88e0-8eabcf7f2b08 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:57.639561 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de009f11-0f83-41eb-88e0-8eabcf7f2b08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:57.640269 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de009f11-0f83-41eb-88e0-8eabcf7f2b08 None admin] RBAC: Adding query filter params (role.id=c8a59d34ee2f4cefa6940cac778260c7, scope.project.id=cd79b6bfb8a449b2a2d0c1f9fe59678f, user.id=bd3d37df64484ce08cbaf6fccb693efa) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:49:57.640602 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de009f11-0f83-41eb-88e0-8eabcf7f2b08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:49:57.642024 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de009f11-0f83-41eb-88e0-8eabcf7f2b08 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:49:57.644786 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:49:57.647507 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 574/1146] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:49:57 2022] GET /identity/v3/role_assignments?user.id=bd3d37df64484ce08cbaf6fccb693efa&scope.project.id=cd79b6bfb8a449b2a2d0c1f9fe59678f&role.id=c8a59d34ee2f4cefa6940cac778260c7 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:49:59.523847 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6f4c0-532a-4696-8029-4c3f27e53649 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:59.524378 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6f4c0-532a-4696-8029-4c3f27e53649 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:59.524378 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6f4c0-532a-4696-8029-4c3f27e53649 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:59.525776 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 573/1147] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:49:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:49:59.531423 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc3c68-17e6-4eca-991b-635e71d2ea20 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:49:59.531827 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc3c68-17e6-4eca-991b-635e71d2ea20 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:49:59.531976 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc3c68-17e6-4eca-991b-635e71d2ea20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:49:59.550639 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-bbbc3c68-17e6-4eca-991b-635e71d2ea20 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:49:59.551822 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-bbbc3c68-17e6-4eca-991b-635e71d2ea20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:49:59.616151 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 575/1148] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:49:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 00:49:59.886608 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6f4c0-532a-4696-8029-4c3f27e53649 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:49:59.987944 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6f4c0-532a-4696-8029-4c3f27e53649 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:00.021070 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ee1c7-9544-4512-8650-877625cff529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:00.021682 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ee1c7-9544-4512-8650-877625cff529 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:00.021957 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ee1c7-9544-4512-8650-877625cff529 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:00.022223 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ee1c7-9544-4512-8650-877625cff529 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:00.032685 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ee1c7-9544-4512-8650-877625cff529 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:50:00.037596 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-c81ee1c7-9544-4512-8650-877625cff529 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:50:00.037596 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:50:00.037596 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:50:00.037596 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:50:00.037596 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:50:00.037596 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:50:00.037596 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:50:00.037596 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:50:00.037596 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:50:00.037596 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:50:00.037596 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:50:00.037596 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:50:00.037596 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:50:00.037596 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:50:00.037596 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:50:00.037596 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:50:00.038721 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:50:00.038721 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:50:00.038721 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:50:00.038721 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:50:00.038721 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:50:00.038721 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:50:00.038721 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:50:00.038721 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:50:00.038721 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:50:00.038721 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:50:00.038721 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:50:00.038721 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:50:00.038721 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:50:00.038721 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:50:00.038721 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:50:00.038721 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:50:00.039562 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:50:00.039562 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:50:00.039562 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:50:00.039562 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:50:00.039562 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:50:00.039562 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:50:00.039562 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:50:00.039562 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:50:00.039562 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:50:00.039562 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:50:00.039562 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:50:00.039562 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:50:00.039562 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 574/1149] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:49:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:50:00.044667 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc3c68-17e6-4eca-991b-635e71d2ea20 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:50:00.141499 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc3c68-17e6-4eca-991b-635e71d2ea20 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:00.169157 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:50:00.174761 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e8725-d77a-4824-bac7-90f4bbb7a53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:00.178814 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e8725-d77a-4824-bac7-90f4bbb7a53c None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:00.178814 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e8725-d77a-4824-bac7-90f4bbb7a53c None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:00.178814 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e8725-d77a-4824-bac7-90f4bbb7a53c None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:00.178814 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e8725-d77a-4824-bac7-90f4bbb7a53c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:50:00.178814 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e8725-d77a-4824-bac7-90f4bbb7a53c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:50:00.178814 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e8725-d77a-4824-bac7-90f4bbb7a53c None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:50:00.186751 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 576/1150] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:50:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:50:00.192505 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ee1c7-9544-4512-8650-877625cff529 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:50:00.235866 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:50:00.291881 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ee1c7-9544-4512-8650-877625cff529 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:00.324472 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d0bbc0-b3af-4ac2-b9e8-f465db121f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:00.325099 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0bbc0-b3af-4ac2-b9e8-f465db121f02 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:00.325377 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0bbc0-b3af-4ac2-b9e8-f465db121f02 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:00.326512 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0bbc0-b3af-4ac2-b9e8-f465db121f02 None admin] PATH_INFO: `/v3/users/bd3d37df64484ce08cbaf6fccb693efa` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:00.327623 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d0bbc0-b3af-4ac2-b9e8-f465db121f02 None admin] RBAC: Authorizing `identity:get_user(user_id=bd3d37df64484ce08cbaf6fccb693efa)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:50:00.329067 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d0bbc0-b3af-4ac2-b9e8-f465db121f02 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:50:00.330624 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 575/1151] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:50:00 2022] GET /identity/v3/users/bd3d37df64484ce08cbaf6fccb693efa => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:50:00.336381 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e8725-d77a-4824-bac7-90f4bbb7a53c None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:50:00.432529 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e8725-d77a-4824-bac7-90f4bbb7a53c None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:00.465012 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ed08f-4454-41e4-9ff3-2d1cfb3d8331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:00.465619 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ed08f-4454-41e4-9ff3-2d1cfb3d8331 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:00.465897 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ed08f-4454-41e4-9ff3-2d1cfb3d8331 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:00.466196 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ed08f-4454-41e4-9ff3-2d1cfb3d8331 None admin] PATH_INFO: `/v3/projects/cd79b6bfb8a449b2a2d0c1f9fe59678f` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:00.467298 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6ed08f-4454-41e4-9ff3-2d1cfb3d8331 None admin] RBAC: Authorizing `identity:get_project(project_id=cd79b6bfb8a449b2a2d0c1f9fe59678f)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:50:00.468740 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6ed08f-4454-41e4-9ff3-2d1cfb3d8331 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:50:00.470312 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 577/1152] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:50:00 2022] GET /identity/v3/projects/cd79b6bfb8a449b2a2d0c1f9fe59678f => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:50:00.476184 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d0bbc0-b3af-4ac2-b9e8-f465db121f02 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:50:00.574519 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d0bbc0-b3af-4ac2-b9e8-f465db121f02 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:00.584584 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:50:00.607483 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840c1f3-2086-4487-8a83-8bd7c7e796e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:00.608130 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840c1f3-2086-4487-8a83-8bd7c7e796e1 None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:00.608414 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840c1f3-2086-4487-8a83-8bd7c7e796e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:00.608724 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840c1f3-2086-4487-8a83-8bd7c7e796e1 None admin] PATH_INFO: `/v3/projects/cd79b6bfb8a449b2a2d0c1f9fe59678f/users/bd3d37df64484ce08cbaf6fccb693efa/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:00.610609 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5840c1f3-2086-4487-8a83-8bd7c7e796e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd79b6bfb8a449b2a2d0c1f9fe59678f, user_id=bd3d37df64484ce08cbaf6fccb693efa, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:50:00.612032 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5840c1f3-2086-4487-8a83-8bd7c7e796e1 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:50:00.716047 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 576/1153] 10.222.0.27 () {58 vars in 1439 bytes} [Sun Sep 4 00:50:00 2022] PUT /identity/v3/projects/cd79b6bfb8a449b2a2d0c1f9fe59678f/users/bd3d37df64484ce08cbaf6fccb693efa/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 00:50:02.617261 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0dfe45-f161-4dcb-9d87-fd648916a2e9 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:02.617805 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0dfe45-f161-4dcb-9d87-fd648916a2e9 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:02.617805 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0dfe45-f161-4dcb-9d87-fd648916a2e9 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:02.619113 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 578/1154] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:50:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:50:02.624955 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd13bba-9bb3-4675-aef1-ee25e5b203bd None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:02.625366 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd13bba-9bb3-4675-aef1-ee25e5b203bd None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:02.625512 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd13bba-9bb3-4675-aef1-ee25e5b203bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:02.644025 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1fd13bba-9bb3-4675-aef1-ee25e5b203bd None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:50:02.645247 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1fd13bba-9bb3-4675-aef1-ee25e5b203bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:50:02.711455 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 577/1155] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:50:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 00:50:02.979496 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0dfe45-f161-4dcb-9d87-fd648916a2e9 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:50:03.038657 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:50:03.079548 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0dfe45-f161-4dcb-9d87-fd648916a2e9 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:03.112435 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349630a0-41a1-4eff-aa1d-9bb093c8662e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:03.113059 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349630a0-41a1-4eff-aa1d-9bb093c8662e None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:03.113336 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349630a0-41a1-4eff-aa1d-9bb093c8662e None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:03.113606 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349630a0-41a1-4eff-aa1d-9bb093c8662e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:03.123758 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349630a0-41a1-4eff-aa1d-9bb093c8662e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:50:03.128473 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-349630a0-41a1-4eff-aa1d-9bb093c8662e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:50:03.128473 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:50:03.128473 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:50:03.128473 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:50:03.128473 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:50:03.128473 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:50:03.128473 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:50:03.128473 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:50:03.128473 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:50:03.128473 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:50:03.128473 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:50:03.128473 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:50:03.128473 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:50:03.128473 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:50:03.128473 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:50:03.128473 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:50:03.129570 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:50:03.129570 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:50:03.129570 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:50:03.129570 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:50:03.129570 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:50:03.129570 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:50:03.129570 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:50:03.129570 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:50:03.129570 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:50:03.129570 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:50:03.129570 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:50:03.129570 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:50:03.129570 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:50:03.129570 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:50:03.129570 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:50:03.129570 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:50:03.130674 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:50:03.130674 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:50:03.130674 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:50:03.130674 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:50:03.130674 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:50:03.130674 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:50:03.130674 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:50:03.130674 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:50:03.130674 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:50:03.130674 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:50:03.130674 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:50:03.130674 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:50:03.130674 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 579/1156] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:50:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:50:03.135522 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd13bba-9bb3-4675-aef1-ee25e5b203bd None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:50:03.228719 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:50:03.232904 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd13bba-9bb3-4675-aef1-ee25e5b203bd None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:03.266076 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81875f-502e-4b73-9242-4bda217e3644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:03.266671 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81875f-502e-4b73-9242-4bda217e3644 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:03.266954 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81875f-502e-4b73-9242-4bda217e3644 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:03.267232 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81875f-502e-4b73-9242-4bda217e3644 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:03.267890 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81875f-502e-4b73-9242-4bda217e3644 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:50:03.268254 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81875f-502e-4b73-9242-4bda217e3644 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:50:03.269611 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81875f-502e-4b73-9242-4bda217e3644 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:50:03.277424 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 578/1157] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:50:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:50:03.283833 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349630a0-41a1-4eff-aa1d-9bb093c8662e None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:50:03.380964 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349630a0-41a1-4eff-aa1d-9bb093c8662e None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:03.399821 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:50:03.414155 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f732f521-37de-4957-884b-fa1c635e9f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:03.414767 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732f521-37de-4957-884b-fa1c635e9f69 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:03.415049 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732f521-37de-4957-884b-fa1c635e9f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:03.415323 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732f521-37de-4957-884b-fa1c635e9f69 None admin] PATH_INFO: `/v3/users/bd3d37df64484ce08cbaf6fccb693efa` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:03.416418 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f732f521-37de-4957-884b-fa1c635e9f69 None admin] RBAC: Authorizing `identity:get_user(user_id=bd3d37df64484ce08cbaf6fccb693efa)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:50:03.417873 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f732f521-37de-4957-884b-fa1c635e9f69 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:50:03.420050 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 580/1158] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:50:03 2022] GET /identity/v3/users/bd3d37df64484ce08cbaf6fccb693efa => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:50:03.426135 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81875f-502e-4b73-9242-4bda217e3644 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:50:03.523387 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81875f-502e-4b73-9242-4bda217e3644 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:03.556119 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77221413-13e6-416c-b1fd-ddcf0f3ec22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:03.556733 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77221413-13e6-416c-b1fd-ddcf0f3ec22f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:03.557011 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77221413-13e6-416c-b1fd-ddcf0f3ec22f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:03.557283 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77221413-13e6-416c-b1fd-ddcf0f3ec22f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:03.586450 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:50:03.623383 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77221413-13e6-416c-b1fd-ddcf0f3ec22f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:50:03.624848 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77221413-13e6-416c-b1fd-ddcf0f3ec22f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:50:03.719505 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 579/1159] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:50:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 04 00:50:03.725335 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f732f521-37de-4957-884b-fa1c635e9f69 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:50:03.822788 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f732f521-37de-4957-884b-fa1c635e9f69 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:03.856204 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a06d38-60f9-4656-8521-430384361dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:03.857026 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a06d38-60f9-4656-8521-430384361dc8 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:03.857414 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a06d38-60f9-4656-8521-430384361dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:03.857697 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a06d38-60f9-4656-8521-430384361dc8 None admin] PATH_INFO: `/v3/projects/cd79b6bfb8a449b2a2d0c1f9fe59678f` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:03.858804 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a06d38-60f9-4656-8521-430384361dc8 None admin] RBAC: Authorizing `identity:get_project(project_id=cd79b6bfb8a449b2a2d0c1f9fe59678f)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:50:03.860228 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a06d38-60f9-4656-8521-430384361dc8 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:50:03.861848 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 581/1160] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:50:03 2022] GET /identity/v3/projects/cd79b6bfb8a449b2a2d0c1f9fe59678f => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:50:03.867822 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77221413-13e6-416c-b1fd-ddcf0f3ec22f None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:50:03.927549 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:50:03.965695 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77221413-13e6-416c-b1fd-ddcf0f3ec22f None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:03.998272 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b85b7fc-249b-4856-bb9d-ef7d5bce09df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:03.998847 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85b7fc-249b-4856-bb9d-ef7d5bce09df None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:03.999125 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85b7fc-249b-4856-bb9d-ef7d5bce09df None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:03.999395 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85b7fc-249b-4856-bb9d-ef7d5bce09df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:04.000097 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b85b7fc-249b-4856-bb9d-ef7d5bce09df None admin] RBAC: Adding query filter params (role.id=c8a59d34ee2f4cefa6940cac778260c7, scope.project.id=cd79b6bfb8a449b2a2d0c1f9fe59678f, user.id=bd3d37df64484ce08cbaf6fccb693efa) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:50:04.000401 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b85b7fc-249b-4856-bb9d-ef7d5bce09df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:50:04.001832 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b85b7fc-249b-4856-bb9d-ef7d5bce09df None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:50:04.007338 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 580/1161] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:50:03 2022] GET /identity/v3/role_assignments?user.id=bd3d37df64484ce08cbaf6fccb693efa&scope.project.id=cd79b6bfb8a449b2a2d0c1f9fe59678f&role.id=c8a59d34ee2f4cefa6940cac778260c7 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:50:05.915241 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9cf42-23aa-4a2f-bdc7-2a652b2f26a2 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:05.915796 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9cf42-23aa-4a2f-bdc7-2a652b2f26a2 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:05.915796 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9cf42-23aa-4a2f-bdc7-2a652b2f26a2 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:05.917177 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 582/1162] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:50:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:50:05.922932 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266202a8-66cc-47ef-8fcf-6fea51157813 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:05.923352 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266202a8-66cc-47ef-8fcf-6fea51157813 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:05.923484 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266202a8-66cc-47ef-8fcf-6fea51157813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:05.942108 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-266202a8-66cc-47ef-8fcf-6fea51157813 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:50:05.943285 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-266202a8-66cc-47ef-8fcf-6fea51157813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:50:06.007637 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 581/1163] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:50:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:50:06.280381 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9cf42-23aa-4a2f-bdc7-2a652b2f26a2 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:50:06.336072 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:50:06.380720 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9cf42-23aa-4a2f-bdc7-2a652b2f26a2 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:06.413536 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9045f04-60e3-44dd-8ae8-952decbe77ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:06.414125 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9045f04-60e3-44dd-8ae8-952decbe77ef None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:06.414407 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9045f04-60e3-44dd-8ae8-952decbe77ef None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:06.414681 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9045f04-60e3-44dd-8ae8-952decbe77ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:06.480732 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9045f04-60e3-44dd-8ae8-952decbe77ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:50:06.482139 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9045f04-60e3-44dd-8ae8-952decbe77ef None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:50:06.534623 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:50:06.578430 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 583/1164] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:50:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 04 00:50:06.584329 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266202a8-66cc-47ef-8fcf-6fea51157813 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:50:06.638401 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:50:06.680615 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266202a8-66cc-47ef-8fcf-6fea51157813 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:06.712589 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c389b-4db0-4058-9561-0271238c515f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:06.713194 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c389b-4db0-4058-9561-0271238c515f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:06.713470 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c389b-4db0-4058-9561-0271238c515f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:06.713736 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c389b-4db0-4058-9561-0271238c515f None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:06.720670 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c389b-4db0-4058-9561-0271238c515f None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:50:06.722093 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c389b-4db0-4058-9561-0271238c515f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:50:06.733151 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-f12c389b-4db0-4058-9561-0271238c515f None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 04 00:50:06.733151 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:50:06.733151 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 04 00:50:06.733151 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 04 00:50:06.733151 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:50:06.733151 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:50:06.733151 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:50:06.733151 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:50:06.733151 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 04 00:50:06.733151 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:50:06.733151 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 00:50:06.733151 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:50:06.733151 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:50:06.733151 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:50:06.733151 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:50:06.733151 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:50:06.733151 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:50:06.734306 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:50:06.734306 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:50:06.734306 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:50:06.734306 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:50:06.734306 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:50:06.734306 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:50:06.734306 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:50:06.734306 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:50:06.734306 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:50:06.734306 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:50:06.734306 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:50:06.734306 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:50:06.734306 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:50:06.734306 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:50:06.734306 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:50:06.734306 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:50:06.735204 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:50:06.735204 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:50:06.735204 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:50:06.735204 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:50:06.735204 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:50:06.735204 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:50:06.735204 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:50:06.735204 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:50:06.735204 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:50:06.735204 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:50:06.735204 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 04 00:50:06.735204 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:50:06.735204 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 04 00:50:06.735204 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:50:06.735204 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 582/1165] 10.222.0.27 () {56 vars in 1117 bytes} [Sun Sep 4 00:50:06 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:50:06.740045 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9045f04-60e3-44dd-8ae8-952decbe77ef None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:50:06.834911 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9045f04-60e3-44dd-8ae8-952decbe77ef None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:06.867719 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211782b3-87d0-4027-b6ea-8154b9fa58e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:06.868305 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211782b3-87d0-4027-b6ea-8154b9fa58e8 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:06.868617 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211782b3-87d0-4027-b6ea-8154b9fa58e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:06.868889 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211782b3-87d0-4027-b6ea-8154b9fa58e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:06.869510 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211782b3-87d0-4027-b6ea-8154b9fa58e8 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:50:06.869807 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211782b3-87d0-4027-b6ea-8154b9fa58e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:50:06.871148 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211782b3-87d0-4027-b6ea-8154b9fa58e8 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:50:06.878805 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 584/1166] 10.222.0.27 () {56 vars in 1115 bytes} [Sun Sep 4 00:50:06 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:50:08.735670 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55713159-240b-4e12-97ca-25ac3808a3b1 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:08.736195 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55713159-240b-4e12-97ca-25ac3808a3b1 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:08.736195 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55713159-240b-4e12-97ca-25ac3808a3b1 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:08.737571 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 583/1167] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:50:08.742650 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06e71b-7723-4b7b-bf0b-78ec53c6fb2b None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:08.743024 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06e71b-7723-4b7b-bf0b-78ec53c6fb2b None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:08.743215 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06e71b-7723-4b7b-bf0b-78ec53c6fb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:08.761603 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-af06e71b-7723-4b7b-bf0b-78ec53c6fb2b None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:50:08.762917 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-af06e71b-7723-4b7b-bf0b-78ec53c6fb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:50:08.776687 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:50:08.828434 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 585/1168] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:50:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 00:50:09.095648 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55713159-240b-4e12-97ca-25ac3808a3b1 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:50:09.195912 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55713159-240b-4e12-97ca-25ac3808a3b1 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:09.211603 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:50:09.229612 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44e089-1d72-4978-94f5-3ffeb1117dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:09.229965 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44e089-1d72-4978-94f5-3ffeb1117dd9 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:09.230248 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44e089-1d72-4978-94f5-3ffeb1117dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:09.230522 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44e089-1d72-4978-94f5-3ffeb1117dd9 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:09.231196 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e44e089-1d72-4978-94f5-3ffeb1117dd9 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:50:09.233361 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e44e089-1d72-4978-94f5-3ffeb1117dd9 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:50:09.320334 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 584/1169] 10.222.0.27 () {60 vars in 1138 bytes} [Sun Sep 4 00:50:09 2022] POST /identity/v3/domains => generated 267 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:50:11.264981 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93262fff-a72a-4264-9345-d732ba97cd19 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:11.265471 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93262fff-a72a-4264-9345-d732ba97cd19 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:11.265568 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93262fff-a72a-4264-9345-d732ba97cd19 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:11.266865 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 586/1170] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:50:11.272042 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78703f5-3759-4ed4-ab65-7434f18e1d23 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:11.272422 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78703f5-3759-4ed4-ab65-7434f18e1d23 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:11.272623 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78703f5-3759-4ed4-ab65-7434f18e1d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:11.290629 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-e78703f5-3759-4ed4-ab65-7434f18e1d23 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:50:11.291804 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e78703f5-3759-4ed4-ab65-7434f18e1d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:50:11.357230 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 585/1171] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:50:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 00:50:11.626971 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93262fff-a72a-4264-9345-d732ba97cd19 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:50:11.725965 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93262fff-a72a-4264-9345-d732ba97cd19 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:11.760024 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad15a1-b935-4a60-8ad6-c83076ef22ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:11.760663 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad15a1-b935-4a60-8ad6-c83076ef22ce None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:11.760941 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad15a1-b935-4a60-8ad6-c83076ef22ce None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:11.761211 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad15a1-b935-4a60-8ad6-c83076ef22ce None admin] PATH_INFO: `/v3/domains/049eb7b41a1249a6bb01c5e6b408c06c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:11.771581 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad15a1-b935-4a60-8ad6-c83076ef22ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=049eb7b41a1249a6bb01c5e6b408c06c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:50:11.773057 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad15a1-b935-4a60-8ad6-c83076ef22ce None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:50:11.774618 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 587/1172] 10.222.0.27 () {56 vars in 1183 bytes} [Sun Sep 4 00:50:11 2022] GET /identity/v3/domains/049eb7b41a1249a6bb01c5e6b408c06c => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:50:11.780386 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78703f5-3759-4ed4-ab65-7434f18e1d23 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:50:11.875797 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78703f5-3759-4ed4-ab65-7434f18e1d23 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:11.882871 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:50:11.908625 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0801f-fa53-4c9c-86bd-1829f29409cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:11.909226 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0801f-fa53-4c9c-86bd-1829f29409cc None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:11.909509 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0801f-fa53-4c9c-86bd-1829f29409cc None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:11.909785 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0801f-fa53-4c9c-86bd-1829f29409cc None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:11.910430 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb0801f-fa53-4c9c-86bd-1829f29409cc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:50:11.912142 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb0801f-fa53-4c9c-86bd-1829f29409cc None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:50:12.074362 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 586/1173] 10.222.0.27 () {60 vars in 1141 bytes} [Sun Sep 4 00:50:11 2022] POST /identity/v3/projects => generated 361 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:50:14.007213 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467f8e6-26b8-40d1-94ff-0bc93cc3c46c None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:14.007795 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467f8e6-26b8-40d1-94ff-0bc93cc3c46c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:14.007795 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467f8e6-26b8-40d1-94ff-0bc93cc3c46c None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:14.009133 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 588/1174] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:50:14.014276 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d7cf0-a53e-4617-94e6-5fe750264fa6 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:14.014696 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d7cf0-a53e-4617-94e6-5fe750264fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:14.014820 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d7cf0-a53e-4617-94e6-5fe750264fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:14.032574 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f17d7cf0-a53e-4617-94e6-5fe750264fa6 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:50:14.033740 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f17d7cf0-a53e-4617-94e6-5fe750264fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:50:14.097590 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 587/1175] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:50:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:50:14.366807 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d467f8e6-26b8-40d1-94ff-0bc93cc3c46c None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:50:14.379685 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:50:14.465485 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d467f8e6-26b8-40d1-94ff-0bc93cc3c46c None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:14.498631 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942030f9-25df-45ef-bffc-3c92bf10bf21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:14.499239 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942030f9-25df-45ef-bffc-3c92bf10bf21 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:14.499516 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942030f9-25df-45ef-bffc-3c92bf10bf21 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:14.499789 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942030f9-25df-45ef-bffc-3c92bf10bf21 None admin] PATH_INFO: `/v3/domains/049eb7b41a1249a6bb01c5e6b408c06c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:14.500921 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942030f9-25df-45ef-bffc-3c92bf10bf21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=049eb7b41a1249a6bb01c5e6b408c06c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:50:14.502345 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942030f9-25df-45ef-bffc-3c92bf10bf21 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:50:14.503852 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 589/1176] 10.222.0.27 () {56 vars in 1183 bytes} [Sun Sep 4 00:50:14 2022] GET /identity/v3/domains/049eb7b41a1249a6bb01c5e6b408c06c => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:50:14.509568 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d7cf0-a53e-4617-94e6-5fe750264fa6 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:50:14.594606 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:50:14.605989 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d7cf0-a53e-4617-94e6-5fe750264fa6 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:14.638986 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d914843d-300c-4add-9467-da06b697cbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:14.639518 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914843d-300c-4add-9467-da06b697cbe8 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:14.639793 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914843d-300c-4add-9467-da06b697cbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:14.640063 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914843d-300c-4add-9467-da06b697cbe8 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:14.640725 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d914843d-300c-4add-9467-da06b697cbe8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:50:14.642125 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d914843d-300c-4add-9467-da06b697cbe8 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:50:14.744225 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 588/1177] 10.222.0.27 () {60 vars in 1132 bytes} [Sun Sep 4 00:50:14 2022] POST /identity/v3/users => generated 311 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:50:16.676915 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5acde7-e79b-435d-99ac-49e5f4c7a935 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:16.677509 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5acde7-e79b-435d-99ac-49e5f4c7a935 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:16.677509 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5acde7-e79b-435d-99ac-49e5f4c7a935 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:16.678764 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 590/1178] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:50:16.683813 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a5531-7a27-47fa-9957-8a2c65c5f9ce None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:16.684760 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a5531-7a27-47fa-9957-8a2c65c5f9ce None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:16.685061 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a5531-7a27-47fa-9957-8a2c65c5f9ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:16.703425 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-2f8a5531-7a27-47fa-9957-8a2c65c5f9ce None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:50:16.704639 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2f8a5531-7a27-47fa-9957-8a2c65c5f9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:50:16.770020 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 589/1179] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:50:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 00:50:17.044424 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5acde7-e79b-435d-99ac-49e5f4c7a935 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:50:17.121004 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:50:17.142512 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5acde7-e79b-435d-99ac-49e5f4c7a935 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:17.175600 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e396f-22d8-47ed-a5be-4337b83385ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:17.176212 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e396f-22d8-47ed-a5be-4337b83385ea None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:17.176518 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e396f-22d8-47ed-a5be-4337b83385ea None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:17.176794 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e396f-22d8-47ed-a5be-4337b83385ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:17.186532 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e396f-22d8-47ed-a5be-4337b83385ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:50:17.191244 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-0e2e396f-22d8-47ed-a5be-4337b83385ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:50:17.191244 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:50:17.191244 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:50:17.191244 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:50:17.191244 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:50:17.191244 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:50:17.191244 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:50:17.191244 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:50:17.191244 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:50:17.191244 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:50:17.191244 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:50:17.191244 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:50:17.191244 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:50:17.191244 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:50:17.191244 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:50:17.191244 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:50:17.192402 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:50:17.192402 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:50:17.192402 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:50:17.192402 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:50:17.192402 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:50:17.192402 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:50:17.192402 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:50:17.192402 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:50:17.192402 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:50:17.192402 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:50:17.192402 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:50:17.192402 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:50:17.192402 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:50:17.192402 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:50:17.192402 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:50:17.192402 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:50:17.193486 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:50:17.193486 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:50:17.193486 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:50:17.193486 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:50:17.193486 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:50:17.193486 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:50:17.193486 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:50:17.193486 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:50:17.193486 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:50:17.193486 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:50:17.193486 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:50:17.193486 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:50:17.193486 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 591/1180] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:50:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:50:17.198232 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a5531-7a27-47fa-9957-8a2c65c5f9ce None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:50:17.270791 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:50:17.295362 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a5531-7a27-47fa-9957-8a2c65c5f9ce None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:17.328284 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67761666-e0ea-4398-8147-fc65b9d6a8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:17.328908 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67761666-e0ea-4398-8147-fc65b9d6a8f3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:17.329190 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67761666-e0ea-4398-8147-fc65b9d6a8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:17.329463 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67761666-e0ea-4398-8147-fc65b9d6a8f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:17.330290 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67761666-e0ea-4398-8147-fc65b9d6a8f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:50:17.330812 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67761666-e0ea-4398-8147-fc65b9d6a8f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:50:17.332589 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67761666-e0ea-4398-8147-fc65b9d6a8f3 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:50:17.340267 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 590/1181] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:50:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:50:17.346678 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e396f-22d8-47ed-a5be-4337b83385ea None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:50:17.443496 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e396f-22d8-47ed-a5be-4337b83385ea None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:17.475954 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e78864-37a1-43d5-8989-e8ae35df7bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:17.476585 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e78864-37a1-43d5-8989-e8ae35df7bc1 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:17.476863 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e78864-37a1-43d5-8989-e8ae35df7bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:17.477133 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e78864-37a1-43d5-8989-e8ae35df7bc1 None admin] PATH_INFO: `/v3/users/cacb08fa0ba34499b8c283bc4b9bd97c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:17.484151 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:50:17.489905 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e78864-37a1-43d5-8989-e8ae35df7bc1 None admin] RBAC: Authorizing `identity:get_user(user_id=cacb08fa0ba34499b8c283bc4b9bd97c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:50:17.491393 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e78864-37a1-43d5-8989-e8ae35df7bc1 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:50:17.492950 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 592/1182] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:50:17 2022] GET /identity/v3/users/cacb08fa0ba34499b8c283bc4b9bd97c => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:50:17.499381 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67761666-e0ea-4398-8147-fc65b9d6a8f3 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:50:17.597279 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67761666-e0ea-4398-8147-fc65b9d6a8f3 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:17.630195 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd94be-28f2-45df-91cb-251ab1e9a18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:17.630780 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd94be-28f2-45df-91cb-251ab1e9a18c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:17.631057 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd94be-28f2-45df-91cb-251ab1e9a18c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:17.631327 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd94be-28f2-45df-91cb-251ab1e9a18c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:17.634549 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:50:17.696007 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cd94be-28f2-45df-91cb-251ab1e9a18c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:50:17.697463 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cd94be-28f2-45df-91cb-251ab1e9a18c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:50:17.792993 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 591/1183] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:50:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 04 00:50:17.798927 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e78864-37a1-43d5-8989-e8ae35df7bc1 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:50:17.895825 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e78864-37a1-43d5-8989-e8ae35df7bc1 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:17.929003 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8402fe7-d3ad-4da6-8ebb-f607d3d12589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:17.929615 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8402fe7-d3ad-4da6-8ebb-f607d3d12589 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:17.929900 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8402fe7-d3ad-4da6-8ebb-f607d3d12589 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:17.930174 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8402fe7-d3ad-4da6-8ebb-f607d3d12589 None admin] PATH_INFO: `/v3/projects/a1c26cddf48a4197bb6948f0461d40c1` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:17.931262 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8402fe7-d3ad-4da6-8ebb-f607d3d12589 None admin] RBAC: Authorizing `identity:get_project(project_id=a1c26cddf48a4197bb6948f0461d40c1)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:50:17.932720 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8402fe7-d3ad-4da6-8ebb-f607d3d12589 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:50:17.934380 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 593/1184] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:50:17 2022] GET /identity/v3/projects/a1c26cddf48a4197bb6948f0461d40c1 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:50:17.940132 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd94be-28f2-45df-91cb-251ab1e9a18c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:50:17.974536 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:50:18.038691 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd94be-28f2-45df-91cb-251ab1e9a18c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:18.071513 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25ef601-20e7-423e-971f-52c066aaa22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:18.072107 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25ef601-20e7-423e-971f-52c066aaa22d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:18.072387 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25ef601-20e7-423e-971f-52c066aaa22d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:18.072687 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25ef601-20e7-423e-971f-52c066aaa22d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:18.073409 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25ef601-20e7-423e-971f-52c066aaa22d None admin] RBAC: Adding query filter params (role.id=c8a59d34ee2f4cefa6940cac778260c7, scope.project.id=a1c26cddf48a4197bb6948f0461d40c1, user.id=cacb08fa0ba34499b8c283bc4b9bd97c) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:50:18.073717 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25ef601-20e7-423e-971f-52c066aaa22d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:50:18.075808 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25ef601-20e7-423e-971f-52c066aaa22d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:50:18.081158 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 592/1185] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:50:17 2022] GET /identity/v3/role_assignments?user.id=cacb08fa0ba34499b8c283bc4b9bd97c&scope.project.id=a1c26cddf48a4197bb6948f0461d40c1&role.id=c8a59d34ee2f4cefa6940cac778260c7 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:50:19.978963 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e6f02-74c4-4c8e-a11a-ce45034ccaca None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:19.979514 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e6f02-74c4-4c8e-a11a-ce45034ccaca None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:19.979514 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e6f02-74c4-4c8e-a11a-ce45034ccaca None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:19.980856 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 594/1186] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:50:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:50:19.986866 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552f683-8e89-4419-b266-d6950ddb87ce None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:19.987301 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552f683-8e89-4419-b266-d6950ddb87ce None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:19.987530 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552f683-8e89-4419-b266-d6950ddb87ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:20.005884 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-9552f683-8e89-4419-b266-d6950ddb87ce None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:50:20.007057 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9552f683-8e89-4419-b266-d6950ddb87ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:50:20.071406 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 593/1187] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:50:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:50:20.338366 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308e6f02-74c4-4c8e-a11a-ce45034ccaca None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:50:20.401354 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:50:20.439756 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308e6f02-74c4-4c8e-a11a-ce45034ccaca None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:20.473268 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388eba1-9e44-476c-86cb-f753ff906c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:20.473886 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388eba1-9e44-476c-86cb-f753ff906c32 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:20.474168 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388eba1-9e44-476c-86cb-f753ff906c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:20.474439 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388eba1-9e44-476c-86cb-f753ff906c32 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:20.485091 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b388eba1-9e44-476c-86cb-f753ff906c32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:50:20.489783 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-b388eba1-9e44-476c-86cb-f753ff906c32 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:50:20.489783 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:50:20.489783 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:50:20.489783 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:50:20.489783 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:50:20.489783 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:50:20.489783 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:50:20.489783 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:50:20.489783 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:50:20.489783 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:50:20.489783 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:50:20.489783 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:50:20.489783 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:50:20.489783 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:50:20.489783 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:50:20.489783 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:50:20.490900 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:50:20.490900 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:50:20.490900 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:50:20.490900 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:50:20.490900 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:50:20.490900 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:50:20.490900 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:50:20.490900 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:50:20.490900 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:50:20.490900 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:50:20.490900 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:50:20.490900 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:50:20.490900 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:50:20.490900 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:50:20.490900 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:50:20.490900 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:50:20.491978 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:50:20.491978 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:50:20.491978 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:50:20.491978 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:50:20.491978 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:50:20.491978 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:50:20.491978 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:50:20.491978 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:50:20.491978 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:50:20.491978 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:50:20.491978 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:50:20.491978 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:50:20.491978 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 595/1188] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:50:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:50:20.496950 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552f683-8e89-4419-b266-d6950ddb87ce None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:50:20.526040 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:50:20.594539 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552f683-8e89-4419-b266-d6950ddb87ce None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:20.627393 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395418e9-39a1-4a55-82cd-4346a50fc60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:20.627989 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395418e9-39a1-4a55-82cd-4346a50fc60b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:20.628271 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395418e9-39a1-4a55-82cd-4346a50fc60b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:20.628573 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395418e9-39a1-4a55-82cd-4346a50fc60b None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:20.629237 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395418e9-39a1-4a55-82cd-4346a50fc60b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:50:20.629529 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395418e9-39a1-4a55-82cd-4346a50fc60b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:50:20.630880 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395418e9-39a1-4a55-82cd-4346a50fc60b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:50:20.638800 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 594/1189] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:50:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:50:20.644514 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388eba1-9e44-476c-86cb-f753ff906c32 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:50:20.742298 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388eba1-9e44-476c-86cb-f753ff906c32 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:20.765295 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:50:20.775727 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209eebf7-eb01-4a32-b840-9c03215951fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:20.784883 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209eebf7-eb01-4a32-b840-9c03215951fc None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:20.784883 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209eebf7-eb01-4a32-b840-9c03215951fc None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:20.784883 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209eebf7-eb01-4a32-b840-9c03215951fc None admin] PATH_INFO: `/v3/users/cacb08fa0ba34499b8c283bc4b9bd97c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:20.784883 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209eebf7-eb01-4a32-b840-9c03215951fc None admin] RBAC: Authorizing `identity:get_user(user_id=cacb08fa0ba34499b8c283bc4b9bd97c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:50:20.784883 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209eebf7-eb01-4a32-b840-9c03215951fc None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:50:20.784883 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 596/1190] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:50:20 2022] GET /identity/v3/users/cacb08fa0ba34499b8c283bc4b9bd97c => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:50:20.787634 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395418e9-39a1-4a55-82cd-4346a50fc60b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:50:20.877878 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:50:20.886341 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395418e9-39a1-4a55-82cd-4346a50fc60b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:20.920284 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc051f28-5bba-4ffe-847d-3f6012cf688a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:20.920927 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc051f28-5bba-4ffe-847d-3f6012cf688a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:20.921210 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc051f28-5bba-4ffe-847d-3f6012cf688a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:20.921486 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc051f28-5bba-4ffe-847d-3f6012cf688a None admin] PATH_INFO: `/v3/projects/a1c26cddf48a4197bb6948f0461d40c1` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:20.922586 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc051f28-5bba-4ffe-847d-3f6012cf688a None admin] RBAC: Authorizing `identity:get_project(project_id=a1c26cddf48a4197bb6948f0461d40c1)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:50:20.924003 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc051f28-5bba-4ffe-847d-3f6012cf688a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:50:20.925636 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 595/1191] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:50:20 2022] GET /identity/v3/projects/a1c26cddf48a4197bb6948f0461d40c1 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:50:20.931591 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209eebf7-eb01-4a32-b840-9c03215951fc None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:50:21.028672 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209eebf7-eb01-4a32-b840-9c03215951fc None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:21.063422 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db2c279-055e-404d-b5d8-7c3cd5dcb368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:21.064070 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2c279-055e-404d-b5d8-7c3cd5dcb368 None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:21.064352 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2c279-055e-404d-b5d8-7c3cd5dcb368 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:21.064673 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2c279-055e-404d-b5d8-7c3cd5dcb368 None admin] PATH_INFO: `/v3/projects/a1c26cddf48a4197bb6948f0461d40c1/users/cacb08fa0ba34499b8c283bc4b9bd97c/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:21.066565 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db2c279-055e-404d-b5d8-7c3cd5dcb368 None admin] RBAC: Authorizing `identity:create_grant(project_id=a1c26cddf48a4197bb6948f0461d40c1, user_id=cacb08fa0ba34499b8c283bc4b9bd97c, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:50:21.068004 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db2c279-055e-404d-b5d8-7c3cd5dcb368 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:50:21.156563 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 597/1192] 10.222.0.27 () {58 vars in 1439 bytes} [Sun Sep 4 00:50:20 2022] PUT /identity/v3/projects/a1c26cddf48a4197bb6948f0461d40c1/users/cacb08fa0ba34499b8c283bc4b9bd97c/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 00:50:23.047492 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d476f0c-a985-4841-85ef-618a17aa8b63 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:23.048043 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d476f0c-a985-4841-85ef-618a17aa8b63 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:23.048043 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d476f0c-a985-4841-85ef-618a17aa8b63 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:23.049426 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 596/1193] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:50:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:50:23.054588 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23777df4-061e-4cd0-8c09-779043e4e9c5 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:23.055012 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23777df4-061e-4cd0-8c09-779043e4e9c5 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:23.055136 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23777df4-061e-4cd0-8c09-779043e4e9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:23.073199 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-23777df4-061e-4cd0-8c09-779043e4e9c5 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:50:23.074381 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-23777df4-061e-4cd0-8c09-779043e4e9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:50:23.120279 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:50:23.140172 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 598/1194] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:50:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:50:23.413412 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d476f0c-a985-4841-85ef-618a17aa8b63 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:50:23.513034 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d476f0c-a985-4841-85ef-618a17aa8b63 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:23.545621 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd5387-378d-4ab9-9e8b-392fecb85db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:23.546232 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd5387-378d-4ab9-9e8b-392fecb85db8 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:23.546515 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd5387-378d-4ab9-9e8b-392fecb85db8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:23.546789 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd5387-378d-4ab9-9e8b-392fecb85db8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:23.556561 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bd5387-378d-4ab9-9e8b-392fecb85db8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:50:23.561887 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-13bd5387-378d-4ab9-9e8b-392fecb85db8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:50:23.561887 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:50:23.561887 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:50:23.561887 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:50:23.561887 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:50:23.561887 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:50:23.561887 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:50:23.561887 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:50:23.561887 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:50:23.561887 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:50:23.561887 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:50:23.561887 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:50:23.561887 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:50:23.561887 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:50:23.561887 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:50:23.561887 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:50:23.563058 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:50:23.563058 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:50:23.563058 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:50:23.563058 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:50:23.563058 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:50:23.563058 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:50:23.563058 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:50:23.563058 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:50:23.563058 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:50:23.563058 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:50:23.563058 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:50:23.563058 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:50:23.563058 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:50:23.563058 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:50:23.563058 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:50:23.563058 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:50:23.564143 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:50:23.564143 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:50:23.564143 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:50:23.564143 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:50:23.564143 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:50:23.564143 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:50:23.564143 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:50:23.564143 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:50:23.564143 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:50:23.564143 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:50:23.564143 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:50:23.564143 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:50:23.564143 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 597/1195] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:50:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:50:23.568985 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23777df4-061e-4cd0-8c09-779043e4e9c5 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:50:23.664130 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23777df4-061e-4cd0-8c09-779043e4e9c5 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:23.697281 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f0ed2-fc00-4437-a899-8ed6ca14ee40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:23.697868 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f0ed2-fc00-4437-a899-8ed6ca14ee40 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:23.698145 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f0ed2-fc00-4437-a899-8ed6ca14ee40 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:23.698416 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f0ed2-fc00-4437-a899-8ed6ca14ee40 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:23.699065 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f0ed2-fc00-4437-a899-8ed6ca14ee40 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:50:23.699364 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f0ed2-fc00-4437-a899-8ed6ca14ee40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:50:23.700750 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f0ed2-fc00-4437-a899-8ed6ca14ee40 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:50:23.708446 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 599/1196] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:50:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:50:23.714358 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd5387-378d-4ab9-9e8b-392fecb85db8 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:50:23.737424 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:50:23.816161 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd5387-378d-4ab9-9e8b-392fecb85db8 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:23.849228 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d9417-7714-4f9e-9245-c2c02dfafaa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:23.849837 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d9417-7714-4f9e-9245-c2c02dfafaa2 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:23.850115 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d9417-7714-4f9e-9245-c2c02dfafaa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:23.850386 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d9417-7714-4f9e-9245-c2c02dfafaa2 None admin] PATH_INFO: `/v3/users/cacb08fa0ba34499b8c283bc4b9bd97c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:23.851481 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10d9417-7714-4f9e-9245-c2c02dfafaa2 None admin] RBAC: Authorizing `identity:get_user(user_id=cacb08fa0ba34499b8c283bc4b9bd97c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:50:23.852928 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10d9417-7714-4f9e-9245-c2c02dfafaa2 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:50:23.854466 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 598/1197] 10.222.0.27 () {56 vars in 1177 bytes} [Sun Sep 4 00:50:23 2022] GET /identity/v3/users/cacb08fa0ba34499b8c283bc4b9bd97c => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:50:23.860976 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f0ed2-fc00-4437-a899-8ed6ca14ee40 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:50:23.905684 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:50:23.959214 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f0ed2-fc00-4437-a899-8ed6ca14ee40 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:23.992268 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745714a5-4317-45b8-a727-14a1f09c92c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:23.992893 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745714a5-4317-45b8-a727-14a1f09c92c1 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:23.993175 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745714a5-4317-45b8-a727-14a1f09c92c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:23.993476 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745714a5-4317-45b8-a727-14a1f09c92c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:24.058311 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745714a5-4317-45b8-a727-14a1f09c92c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:50:24.059749 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745714a5-4317-45b8-a727-14a1f09c92c1 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:50:24.102125 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:50:24.155212 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 600/1198] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:50:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 04 00:50:24.161493 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d9417-7714-4f9e-9245-c2c02dfafaa2 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:50:24.249450 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:50:24.260682 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d9417-7714-4f9e-9245-c2c02dfafaa2 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:24.294223 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa4412-64f1-4808-ac4e-d0493d4a4628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:24.295416 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa4412-64f1-4808-ac4e-d0493d4a4628 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:24.295696 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa4412-64f1-4808-ac4e-d0493d4a4628 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:24.295972 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa4412-64f1-4808-ac4e-d0493d4a4628 None admin] PATH_INFO: `/v3/projects/a1c26cddf48a4197bb6948f0461d40c1` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:24.297136 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa4412-64f1-4808-ac4e-d0493d4a4628 None admin] RBAC: Authorizing `identity:get_project(project_id=a1c26cddf48a4197bb6948f0461d40c1)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:50:24.299578 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa4412-64f1-4808-ac4e-d0493d4a4628 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:50:24.301206 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 599/1199] 10.222.0.27 () {56 vars in 1186 bytes} [Sun Sep 4 00:50:24 2022] GET /identity/v3/projects/a1c26cddf48a4197bb6948f0461d40c1 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:50:24.307393 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745714a5-4317-45b8-a727-14a1f09c92c1 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:50:24.403206 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745714a5-4317-45b8-a727-14a1f09c92c1 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:50:24.436818 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3a8b3-ef1c-4344-bc4d-e14c34df1489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:50:24.437406 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3a8b3-ef1c-4344-bc4d-e14c34df1489 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:50:24.437688 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3a8b3-ef1c-4344-bc4d-e14c34df1489 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:50:24.437960 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3a8b3-ef1c-4344-bc4d-e14c34df1489 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:50:24.438667 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e3a8b3-ef1c-4344-bc4d-e14c34df1489 None admin] RBAC: Adding query filter params (role.id=c8a59d34ee2f4cefa6940cac778260c7, scope.project.id=a1c26cddf48a4197bb6948f0461d40c1, user.id=cacb08fa0ba34499b8c283bc4b9bd97c) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:50:24.438971 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e3a8b3-ef1c-4344-bc4d-e14c34df1489 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:50:24.440387 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e3a8b3-ef1c-4344-bc4d-e14c34df1489 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:50:24.445947 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 601/1200] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:50:24 2022] GET /identity/v3/role_assignments?user.id=cacb08fa0ba34499b8c283bc4b9bd97c&scope.project.id=a1c26cddf48a4197bb6948f0461d40c1&role.id=c8a59d34ee2f4cefa6940cac778260c7 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:50:34.290006 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=22 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:50:34.443400 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=96 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:35.679447 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d9730-08a5-4a03-abd8-906c864a2716 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:35.680081 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d9730-08a5-4a03-abd8-906c864a2716 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:35.680081 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d9730-08a5-4a03-abd8-906c864a2716 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:35.681360 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 600/1201] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:51:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:51:35.686651 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c26405b-9eb2-4c5b-a52e-db4286a81c44 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:35.687145 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c26405b-9eb2-4c5b-a52e-db4286a81c44 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:35.687241 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c26405b-9eb2-4c5b-a52e-db4286a81c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:35.706073 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-1c26405b-9eb2-4c5b-a52e-db4286a81c44 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:51:35.707714 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-1c26405b-9eb2-4c5b-a52e-db4286a81c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:51:35.772732 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 602/1202] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:51:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:51:36.048250 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d9730-08a5-4a03-abd8-906c864a2716 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:36.148322 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d9730-08a5-4a03-abd8-906c864a2716 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:36.183298 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96929d87-d7c8-4cc1-999f-8dab433a01d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:36.183903 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96929d87-d7c8-4cc1-999f-8dab433a01d5 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:36.184180 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96929d87-d7c8-4cc1-999f-8dab433a01d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:36.184450 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96929d87-d7c8-4cc1-999f-8dab433a01d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:36.195302 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96929d87-d7c8-4cc1-999f-8dab433a01d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:36.198134 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96929d87-d7c8-4cc1-999f-8dab433a01d5 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:36.211860 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-96929d87-d7c8-4cc1-999f-8dab433a01d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:51:36.211860 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:51:36.211860 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:51:36.211860 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:51:36.211860 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:51:36.211860 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:51:36.211860 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:51:36.211860 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:51:36.211860 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:51:36.211860 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:51:36.211860 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:51:36.211860 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:51:36.211860 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:51:36.211860 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:51:36.211860 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:51:36.211860 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:51:36.213065 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:51:36.213065 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:51:36.213065 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:51:36.213065 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:51:36.213065 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:51:36.213065 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:51:36.213065 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:51:36.213065 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:51:36.213065 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:51:36.213065 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:51:36.213065 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:51:36.213065 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:51:36.213065 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:51:36.213065 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:51:36.213065 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:51:36.213065 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:51:36.213065 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:51:36.214222 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:51:36.214222 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:51:36.214222 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:51:36.214222 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 601/1203] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:51:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:51:36.219051 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c26405b-9eb2-4c5b-a52e-db4286a81c44 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:36.318917 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c26405b-9eb2-4c5b-a52e-db4286a81c44 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:36.346821 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:36.352284 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880d497-75ee-420e-8a10-995c56bd5421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:36.352942 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880d497-75ee-420e-8a10-995c56bd5421 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:36.353227 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880d497-75ee-420e-8a10-995c56bd5421 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:36.353502 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880d497-75ee-420e-8a10-995c56bd5421 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:36.354135 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b880d497-75ee-420e-8a10-995c56bd5421 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:51:36.354438 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b880d497-75ee-420e-8a10-995c56bd5421 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:36.355802 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b880d497-75ee-420e-8a10-995c56bd5421 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:36.367468 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 603/1204] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:51:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:51:36.373459 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96929d87-d7c8-4cc1-999f-8dab433a01d5 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:36.426913 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:36.474220 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96929d87-d7c8-4cc1-999f-8dab433a01d5 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:36.508325 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee5722-0af1-4ba2-8ece-ed1735fef6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:36.508983 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee5722-0af1-4ba2-8ece-ed1735fef6ce None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:36.509258 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee5722-0af1-4ba2-8ece-ed1735fef6ce None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:36.509531 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee5722-0af1-4ba2-8ece-ed1735fef6ce None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:36.510190 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee5722-0af1-4ba2-8ece-ed1735fef6ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:36.511624 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee5722-0af1-4ba2-8ece-ed1735fef6ce None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:36.588400 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 602/1205] 10.222.0.27 () {60 vars in 1132 bytes} [Sun Sep 4 00:51:36 2022] POST /identity/v3/users => generated 251 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:51:38.520038 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd7e4e-7cdf-4ae8-b3d5-995b411fb55d None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:38.520645 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd7e4e-7cdf-4ae8-b3d5-995b411fb55d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:38.520645 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd7e4e-7cdf-4ae8-b3d5-995b411fb55d None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:38.521940 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 604/1206] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:51:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:51:38.527067 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ffbc2-8488-4f76-88ba-77cf0fa746ee None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:38.527438 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ffbc2-8488-4f76-88ba-77cf0fa746ee None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:38.527616 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ffbc2-8488-4f76-88ba-77cf0fa746ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:38.545961 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-527ffbc2-8488-4f76-88ba-77cf0fa746ee None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:51:38.547138 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-527ffbc2-8488-4f76-88ba-77cf0fa746ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:51:38.613850 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 603/1207] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:51:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:51:38.885405 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd7e4e-7cdf-4ae8-b3d5-995b411fb55d None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:38.987619 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd7e4e-7cdf-4ae8-b3d5-995b411fb55d None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:39.022031 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c4191b-ceb5-404b-9a19-a756deb171d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:39.022651 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4191b-ceb5-404b-9a19-a756deb171d7 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:39.022934 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4191b-ceb5-404b-9a19-a756deb171d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:39.023205 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4191b-ceb5-404b-9a19-a756deb171d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:39.033109 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c4191b-ceb5-404b-9a19-a756deb171d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:51:39.038689 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-b0c4191b-ceb5-404b-9a19-a756deb171d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:51:39.038689 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:51:39.038689 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:51:39.038689 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:51:39.038689 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:51:39.038689 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:51:39.038689 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:51:39.038689 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:51:39.038689 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:51:39.038689 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:51:39.038689 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:51:39.038689 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:51:39.038689 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:51:39.038689 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:51:39.038689 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:51:39.038689 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:51:39.039825 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:51:39.039825 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:51:39.039825 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:51:39.039825 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:51:39.039825 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:51:39.039825 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:51:39.039825 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:51:39.039825 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:51:39.039825 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:51:39.039825 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:51:39.039825 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:51:39.039825 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:51:39.039825 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:51:39.039825 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:51:39.039825 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:51:39.039825 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:51:39.040896 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:51:39.040896 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:51:39.040896 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:51:39.040896 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:51:39.040896 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:51:39.040896 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:51:39.040896 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:51:39.040896 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:51:39.040896 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:51:39.040896 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:51:39.040896 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:51:39.040896 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:51:39.040896 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 605/1208] 10.222.0.27 () {56 vars in 1102 bytes} [Sun Sep 4 00:51:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:51:39.046085 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ffbc2-8488-4f76-88ba-77cf0fa746ee None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:39.087140 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:39.147601 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ffbc2-8488-4f76-88ba-77cf0fa746ee None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:39.182787 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433583a-5dc5-410c-8145-ade078272c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:39.183379 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433583a-5dc5-410c-8145-ade078272c31 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:39.183659 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433583a-5dc5-410c-8145-ade078272c31 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:39.183930 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433583a-5dc5-410c-8145-ade078272c31 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:39.184619 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433583a-5dc5-410c-8145-ade078272c31 None admin] RBAC: Adding query filter params (name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:51:39.184919 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433583a-5dc5-410c-8145-ade078272c31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:39.186271 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433583a-5dc5-410c-8145-ade078272c31 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:39.194292 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 604/1209] 10.222.0.27 () {56 vars in 1103 bytes} [Sun Sep 4 00:51:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:51:39.200230 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c4191b-ceb5-404b-9a19-a756deb171d7 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:39.253393 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:39.302188 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c4191b-ceb5-404b-9a19-a756deb171d7 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:39.336147 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597a8e4-9ce9-4e7f-86a5-81f821740c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:39.336791 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597a8e4-9ce9-4e7f-86a5-81f821740c96 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:39.337071 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597a8e4-9ce9-4e7f-86a5-81f821740c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:39.337346 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597a8e4-9ce9-4e7f-86a5-81f821740c96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:39.347990 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d597a8e4-9ce9-4e7f-86a5-81f821740c96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:39.349467 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d597a8e4-9ce9-4e7f-86a5-81f821740c96 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:39.363965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-d597a8e4-9ce9-4e7f-86a5-81f821740c96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:51:39.363965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:51:39.363965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:51:39.363965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:51:39.363965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:51:39.363965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:51:39.363965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:51:39.363965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:51:39.363965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:51:39.363965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:51:39.363965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:51:39.363965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:51:39.363965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:51:39.363965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:51:39.363965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:51:39.363965 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:51:39.365134 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:51:39.365134 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:51:39.365134 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:51:39.365134 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:51:39.365134 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:51:39.365134 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:51:39.365134 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:51:39.365134 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:51:39.365134 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:51:39.365134 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:51:39.365134 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:51:39.365134 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:51:39.365134 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:51:39.365134 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:51:39.365134 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:51:39.365134 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:51:39.365134 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:51:39.366223 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:51:39.366223 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:51:39.366223 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:51:39.366223 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 606/1210] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:51:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:51:39.371089 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433583a-5dc5-410c-8145-ade078272c31 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:39.466675 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:39.469852 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433583a-5dc5-410c-8145-ade078272c31 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:39.502888 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f183af-ecc3-475c-89e6-c76f74645e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:39.503460 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f183af-ecc3-475c-89e6-c76f74645e1b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:39.503737 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f183af-ecc3-475c-89e6-c76f74645e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:39.504009 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f183af-ecc3-475c-89e6-c76f74645e1b None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:39.504668 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f183af-ecc3-475c-89e6-c76f74645e1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:51:39.504972 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f183af-ecc3-475c-89e6-c76f74645e1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:39.506306 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f183af-ecc3-475c-89e6-c76f74645e1b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:39.517167 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 605/1211] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:51:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:51:39.523405 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597a8e4-9ce9-4e7f-86a5-81f821740c96 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:39.623840 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597a8e4-9ce9-4e7f-86a5-81f821740c96 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:39.625385 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:39.658821 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201797a0-0529-4c87-b1a2-461a2bb6b828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:39.659985 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201797a0-0529-4c87-b1a2-461a2bb6b828 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:39.660342 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201797a0-0529-4c87-b1a2-461a2bb6b828 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:39.660682 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201797a0-0529-4c87-b1a2-461a2bb6b828 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:39.668553 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201797a0-0529-4c87-b1a2-461a2bb6b828 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:39.670073 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201797a0-0529-4c87-b1a2-461a2bb6b828 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:39.682465 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-201797a0-0529-4c87-b1a2-461a2bb6b828 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 04 00:51:39.682465 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:51:39.682465 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:51:39.682465 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:51:39.682465 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:51:39.682465 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:51:39.682465 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:51:39.682465 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:51:39.682465 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:51:39.682465 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:51:39.682465 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:51:39.682465 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:51:39.682465 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:51:39.682465 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:51:39.682465 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:51:39.682465 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:51:39.683642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:51:39.683642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:51:39.683642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:51:39.683642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:51:39.683642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:51:39.683642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:51:39.683642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:51:39.683642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:51:39.683642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:51:39.683642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:51:39.683642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:51:39.683642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:51:39.683642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:51:39.683642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:51:39.683642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:51:39.683642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:51:39.683642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:51:39.683642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:51:39.684916 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:51:39.684916 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:51:39.684916 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:51:39.684916 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:51:39.684916 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:51:39.684916 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:51:39.684916 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:51:39.684916 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:51:39.684916 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 04 00:51:39.684916 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:51:39.684916 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 607/1212] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:51:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:51:39.690571 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f183af-ecc3-475c-89e6-c76f74645e1b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:39.793848 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f183af-ecc3-475c-89e6-c76f74645e1b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:39.827390 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57189374-ac81-4087-824f-f13750a5ff6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:39.827993 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57189374-ac81-4087-824f-f13750a5ff6c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:39.828274 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57189374-ac81-4087-824f-f13750a5ff6c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:39.828579 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57189374-ac81-4087-824f-f13750a5ff6c None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:39.829301 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57189374-ac81-4087-824f-f13750a5ff6c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:51:39.829608 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57189374-ac81-4087-824f-f13750a5ff6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:39.831581 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57189374-ac81-4087-824f-f13750a5ff6c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:39.848327 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 606/1213] 10.222.0.27 () {56 vars in 1143 bytes} [Sun Sep 4 00:51:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:51:39.854232 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201797a0-0529-4c87-b1a2-461a2bb6b828 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:39.957087 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201797a0-0529-4c87-b1a2-461a2bb6b828 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:39.991711 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13a30c-8f15-413d-8e43-eebaba3ec22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:39.992309 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13a30c-8f15-413d-8e43-eebaba3ec22f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:39.992627 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13a30c-8f15-413d-8e43-eebaba3ec22f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:39.992906 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13a30c-8f15-413d-8e43-eebaba3ec22f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:40.027343 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:40.066835 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13a30c-8f15-413d-8e43-eebaba3ec22f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:40.068258 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13a30c-8f15-413d-8e43-eebaba3ec22f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:40.165150 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 608/1214] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:51:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 04 00:51:40.171124 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57189374-ac81-4087-824f-f13750a5ff6c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:40.191312 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:40.272043 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57189374-ac81-4087-824f-f13750a5ff6c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:40.305945 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d6e150-2d97-48b8-8f5e-579cc59ab132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:40.306563 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d6e150-2d97-48b8-8f5e-579cc59ab132 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:40.306842 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d6e150-2d97-48b8-8f5e-579cc59ab132 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:40.307168 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d6e150-2d97-48b8-8f5e-579cc59ab132 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:40.318780 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d6e150-2d97-48b8-8f5e-579cc59ab132 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:40.320146 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d6e150-2d97-48b8-8f5e-579cc59ab132 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:40.334737 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-d6d6e150-2d97-48b8-8f5e-579cc59ab132 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:51:40.334737 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:51:40.334737 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:51:40.334737 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:51:40.334737 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:51:40.334737 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:51:40.334737 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:51:40.334737 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:51:40.334737 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:51:40.334737 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:51:40.334737 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:51:40.334737 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:51:40.334737 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:51:40.334737 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:51:40.334737 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:51:40.334737 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:51:40.335864 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:51:40.335864 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:51:40.335864 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:51:40.335864 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:51:40.335864 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:51:40.335864 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:51:40.335864 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:51:40.335864 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:51:40.335864 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:51:40.335864 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:51:40.335864 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:51:40.335864 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:51:40.335864 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:51:40.335864 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:51:40.335864 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:51:40.335864 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:51:40.335864 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:51:40.336948 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:51:40.336948 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:51:40.336948 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:51:40.340695 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 607/1215] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:51:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:51:40.341726 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13a30c-8f15-413d-8e43-eebaba3ec22f None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:40.401433 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:40.440219 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13a30c-8f15-413d-8e43-eebaba3ec22f None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:40.473120 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a42c5b-34b0-4160-9d61-777f9f0a01bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:40.473720 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a42c5b-34b0-4160-9d61-777f9f0a01bf None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:40.473999 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a42c5b-34b0-4160-9d61-777f9f0a01bf None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:40.474266 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a42c5b-34b0-4160-9d61-777f9f0a01bf None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:40.474891 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a42c5b-34b0-4160-9d61-777f9f0a01bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:51:40.475190 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a42c5b-34b0-4160-9d61-777f9f0a01bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:40.476579 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a42c5b-34b0-4160-9d61-777f9f0a01bf None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:40.488642 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 609/1216] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:51:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:51:40.494589 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d6e150-2d97-48b8-8f5e-579cc59ab132 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:40.567158 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:40.597645 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d6e150-2d97-48b8-8f5e-579cc59ab132 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:40.630664 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d7de9-ad59-41e3-9f15-5108dccac004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:40.631287 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d7de9-ad59-41e3-9f15-5108dccac004 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:40.631565 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d7de9-ad59-41e3-9f15-5108dccac004 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:40.631836 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d7de9-ad59-41e3-9f15-5108dccac004 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:40.638302 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855d7de9-ad59-41e3-9f15-5108dccac004 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:40.639783 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855d7de9-ad59-41e3-9f15-5108dccac004 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:40.651483 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-855d7de9-ad59-41e3-9f15-5108dccac004 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:51:40.651483 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:51:40.651483 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:51:40.651483 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:51:40.651483 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:51:40.651483 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:51:40.651483 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:51:40.651483 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:51:40.651483 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:51:40.651483 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:51:40.651483 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:51:40.651483 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:51:40.651483 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:51:40.651483 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:51:40.651483 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:51:40.652594 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:51:40.652594 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:51:40.652594 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:51:40.652594 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:51:40.652594 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:51:40.652594 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:51:40.652594 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:51:40.652594 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:51:40.652594 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:51:40.652594 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:51:40.652594 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:51:40.652594 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:51:40.652594 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:51:40.652594 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:51:40.652594 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:51:40.652594 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:51:40.652594 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:51:40.653819 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:51:40.653819 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:51:40.653819 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:51:40.653819 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:51:40.653819 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:51:40.653819 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:51:40.653819 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:51:40.653819 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:51:40.653819 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 608/1217] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:51:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:51:40.659486 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a42c5b-34b0-4160-9d61-777f9f0a01bf None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:40.758503 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a42c5b-34b0-4160-9d61-777f9f0a01bf None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:40.781042 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:40.793522 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa61b4-1366-432e-b1b9-20eda33d3d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:40.794129 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa61b4-1366-432e-b1b9-20eda33d3d02 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:40.794413 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa61b4-1366-432e-b1b9-20eda33d3d02 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:40.794986 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa61b4-1366-432e-b1b9-20eda33d3d02 None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:40.795370 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaa61b4-1366-432e-b1b9-20eda33d3d02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:51:40.795683 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaa61b4-1366-432e-b1b9-20eda33d3d02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:40.797164 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaa61b4-1366-432e-b1b9-20eda33d3d02 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:40.808516 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 610/1218] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:51:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:51:40.814891 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d7de9-ad59-41e3-9f15-5108dccac004 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:40.915248 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d7de9-ad59-41e3-9f15-5108dccac004 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:40.942802 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:40.948557 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a30b33-e2fd-4f2d-a06f-3a47acb8436b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:40.949153 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a30b33-e2fd-4f2d-a06f-3a47acb8436b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:40.949436 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a30b33-e2fd-4f2d-a06f-3a47acb8436b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:40.949712 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a30b33-e2fd-4f2d-a06f-3a47acb8436b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:40.950426 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a30b33-e2fd-4f2d-a06f-3a47acb8436b None admin] RBAC: Adding query filter params (role.id=6e3e398ba29e436e8532a45d2fdf1321, scope.project.id=5100a8d316d243fb8f82e9044e1b3b06, user.id=018802a8d9c041119ad708430f885a89) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:51:40.950738 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a30b33-e2fd-4f2d-a06f-3a47acb8436b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:40.952181 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a30b33-e2fd-4f2d-a06f-3a47acb8436b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:40.958045 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 609/1219] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:51:40 2022] GET /identity/v3/role_assignments?user.id=018802a8d9c041119ad708430f885a89&scope.project.id=5100a8d316d243fb8f82e9044e1b3b06&role.id=6e3e398ba29e436e8532a45d2fdf1321 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:51:42.855784 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24277ca7-b85c-42d6-8045-12e527a1a89b None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:42.856313 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24277ca7-b85c-42d6-8045-12e527a1a89b None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:42.856313 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24277ca7-b85c-42d6-8045-12e527a1a89b None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:42.857810 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 611/1220] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:51:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:51:42.863192 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2351cc7-fad1-4e81-80b2-6050fd57a2b5 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:42.863598 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2351cc7-fad1-4e81-80b2-6050fd57a2b5 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:42.863776 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2351cc7-fad1-4e81-80b2-6050fd57a2b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:42.882415 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-a2351cc7-fad1-4e81-80b2-6050fd57a2b5 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:51:42.883595 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a2351cc7-fad1-4e81-80b2-6050fd57a2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:51:42.950736 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 610/1221] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:51:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 00:51:43.235611 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24277ca7-b85c-42d6-8045-12e527a1a89b None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:43.339305 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24277ca7-b85c-42d6-8045-12e527a1a89b None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:43.371852 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77fd599-3fdd-4c74-bdbe-116f2e6ebbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:43.372490 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77fd599-3fdd-4c74-bdbe-116f2e6ebbdc None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:43.372770 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77fd599-3fdd-4c74-bdbe-116f2e6ebbdc None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:43.373042 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77fd599-3fdd-4c74-bdbe-116f2e6ebbdc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:43.384011 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77fd599-3fdd-4c74-bdbe-116f2e6ebbdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:51:43.388781 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-f77fd599-3fdd-4c74-bdbe-116f2e6ebbdc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:51:43.388781 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:51:43.388781 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:51:43.388781 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:51:43.388781 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:51:43.388781 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:51:43.388781 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:51:43.388781 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:51:43.388781 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:51:43.388781 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:51:43.388781 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:51:43.388781 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:51:43.388781 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:51:43.388781 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:51:43.388781 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:51:43.388781 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:51:43.390014 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:51:43.390014 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:51:43.390014 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:51:43.390014 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:51:43.390014 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:51:43.390014 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:51:43.390014 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:51:43.390014 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:51:43.390014 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:51:43.390014 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:51:43.390014 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:51:43.390014 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:51:43.390014 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:51:43.390014 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:51:43.390014 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:51:43.390014 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:51:43.390991 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:51:43.390991 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:51:43.390991 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:51:43.390991 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:51:43.390991 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:51:43.390991 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:51:43.390991 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:51:43.390991 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:51:43.390991 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:51:43.390991 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:51:43.390991 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:51:43.390991 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:51:43.390991 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 612/1222] 10.222.0.27 () {56 vars in 1102 bytes} [Sun Sep 4 00:51:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:51:43.396872 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2351cc7-fad1-4e81-80b2-6050fd57a2b5 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:43.494054 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2351cc7-fad1-4e81-80b2-6050fd57a2b5 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:43.517170 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:43.528262 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d0b53b-90e3-4a70-bbc5-282976813c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:43.528893 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d0b53b-90e3-4a70-bbc5-282976813c28 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:43.529173 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d0b53b-90e3-4a70-bbc5-282976813c28 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:43.529487 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d0b53b-90e3-4a70-bbc5-282976813c28 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:43.530141 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d0b53b-90e3-4a70-bbc5-282976813c28 None admin] RBAC: Adding query filter params (name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:51:43.530448 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d0b53b-90e3-4a70-bbc5-282976813c28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:43.531802 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d0b53b-90e3-4a70-bbc5-282976813c28 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:43.540349 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 611/1223] 10.222.0.27 () {56 vars in 1103 bytes} [Sun Sep 4 00:51:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:51:43.546253 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77fd599-3fdd-4c74-bdbe-116f2e6ebbdc None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:43.590896 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:43.647370 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77fd599-3fdd-4c74-bdbe-116f2e6ebbdc None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:43.681831 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc0d6c-417e-49ad-ab9e-ba7402572d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:43.682455 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc0d6c-417e-49ad-ab9e-ba7402572d31 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:43.682745 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc0d6c-417e-49ad-ab9e-ba7402572d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:43.683020 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc0d6c-417e-49ad-ab9e-ba7402572d31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:43.694156 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc0d6c-417e-49ad-ab9e-ba7402572d31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:43.695647 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc0d6c-417e-49ad-ab9e-ba7402572d31 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:43.710213 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-80bc0d6c-417e-49ad-ab9e-ba7402572d31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:51:43.710213 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:51:43.710213 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:51:43.710213 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:51:43.710213 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:51:43.710213 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:51:43.710213 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:51:43.710213 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:51:43.710213 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:51:43.710213 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:51:43.710213 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:51:43.710213 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:51:43.710213 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:51:43.710213 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:51:43.710213 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:51:43.710213 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:51:43.711299 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:51:43.711299 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:51:43.711299 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:51:43.711299 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:51:43.711299 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:51:43.711299 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:51:43.711299 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:51:43.711299 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:51:43.711299 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:51:43.711299 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:51:43.711299 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:51:43.711299 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:51:43.711299 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:51:43.711299 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:51:43.711299 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:51:43.711299 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:51:43.711299 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:51:43.712199 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:51:43.712199 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:51:43.712199 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:51:43.712199 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 613/1224] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:51:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:51:43.717176 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d0b53b-90e3-4a70-bbc5-282976813c28 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:43.817347 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d0b53b-90e3-4a70-bbc5-282976813c28 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:43.851167 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e3918b-8d69-4bad-a94a-cf0edbb25893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:43.851764 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e3918b-8d69-4bad-a94a-cf0edbb25893 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:43.852047 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e3918b-8d69-4bad-a94a-cf0edbb25893 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:43.852322 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e3918b-8d69-4bad-a94a-cf0edbb25893 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:43.852986 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e3918b-8d69-4bad-a94a-cf0edbb25893 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:51:43.853290 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e3918b-8d69-4bad-a94a-cf0edbb25893 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:43.854714 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e3918b-8d69-4bad-a94a-cf0edbb25893 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:43.867071 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 612/1225] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:51:43.872772 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc0d6c-417e-49ad-ab9e-ba7402572d31 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:43.963707 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:43.970308 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc0d6c-417e-49ad-ab9e-ba7402572d31 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:44.004357 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1840898-87e9-4a09-920b-bf805bac99b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:44.005012 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1840898-87e9-4a09-920b-bf805bac99b3 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:44.005291 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1840898-87e9-4a09-920b-bf805bac99b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:44.005566 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1840898-87e9-4a09-920b-bf805bac99b3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:44.012348 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1840898-87e9-4a09-920b-bf805bac99b3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:44.013834 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1840898-87e9-4a09-920b-bf805bac99b3 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:44.025164 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-d1840898-87e9-4a09-920b-bf805bac99b3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 04 00:51:44.025164 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:51:44.025164 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:51:44.025164 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:51:44.025164 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:51:44.025164 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:51:44.025164 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:51:44.025164 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:51:44.025164 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:51:44.025164 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:51:44.025164 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:51:44.025164 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:51:44.025164 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:51:44.025164 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:51:44.025164 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:51:44.025164 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:51:44.026157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:51:44.026157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:51:44.026157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:51:44.026157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:51:44.026157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:51:44.026157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:51:44.026157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:51:44.026157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:51:44.026157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:51:44.026157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:51:44.026157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:51:44.026157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:51:44.026157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:51:44.026157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:51:44.026157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:51:44.026157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:51:44.026157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:51:44.026157 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:51:44.027115 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:51:44.027115 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:51:44.027115 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:51:44.027115 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:51:44.027115 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:51:44.027115 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:51:44.027115 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:51:44.027115 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:51:44.027115 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 04 00:51:44.027115 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:51:44.027115 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 614/1226] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:51:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:51:44.032248 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e3918b-8d69-4bad-a94a-cf0edbb25893 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:44.079549 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:44.130314 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e3918b-8d69-4bad-a94a-cf0edbb25893 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:44.162982 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0720266-659c-4227-87e3-d8e0d937dfb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:44.163576 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0720266-659c-4227-87e3-d8e0d937dfb2 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:44.163852 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0720266-659c-4227-87e3-d8e0d937dfb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:44.164128 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0720266-659c-4227-87e3-d8e0d937dfb2 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:44.164882 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0720266-659c-4227-87e3-d8e0d937dfb2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:51:44.165394 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0720266-659c-4227-87e3-d8e0d937dfb2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:44.167623 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0720266-659c-4227-87e3-d8e0d937dfb2 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:44.184801 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 613/1227] 10.222.0.27 () {56 vars in 1143 bytes} [Sun Sep 4 00:51:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:51:44.190648 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1840898-87e9-4a09-920b-bf805bac99b3 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:44.290966 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1840898-87e9-4a09-920b-bf805bac99b3 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:44.325061 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f080b8-1792-4639-98a0-df2875452b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:44.325681 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f080b8-1792-4639-98a0-df2875452b9c None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:44.325963 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f080b8-1792-4639-98a0-df2875452b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:44.326240 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f080b8-1792-4639-98a0-df2875452b9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:44.340700 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f080b8-1792-4639-98a0-df2875452b9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:44.340700 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f080b8-1792-4639-98a0-df2875452b9c None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:44.355544 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-04f080b8-1792-4639-98a0-df2875452b9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:51:44.355544 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:51:44.355544 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:51:44.355544 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:51:44.355544 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:51:44.355544 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:51:44.355544 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:51:44.355544 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:51:44.355544 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:51:44.355544 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:51:44.355544 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:51:44.355544 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:51:44.355544 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:51:44.355544 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:51:44.355544 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:51:44.355544 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:51:44.356747 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:51:44.356747 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:51:44.356747 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:51:44.356747 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:51:44.356747 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:51:44.356747 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:51:44.356747 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:51:44.356747 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:51:44.356747 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:51:44.356747 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:51:44.356747 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:51:44.356747 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:51:44.356747 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:51:44.356747 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:51:44.356747 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:51:44.356747 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:51:44.356747 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:51:44.357868 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:51:44.357868 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:51:44.357868 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:51:44.357868 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 615/1228] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:51:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:51:44.363284 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0720266-659c-4227-87e3-d8e0d937dfb2 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:44.461561 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0720266-659c-4227-87e3-d8e0d937dfb2 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:44.463821 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:44.495691 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eee552-1b5d-4f1c-8dd7-137706d1e9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:44.496296 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eee552-1b5d-4f1c-8dd7-137706d1e9fb None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:44.496611 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eee552-1b5d-4f1c-8dd7-137706d1e9fb None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:44.496888 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eee552-1b5d-4f1c-8dd7-137706d1e9fb None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:44.497519 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eee552-1b5d-4f1c-8dd7-137706d1e9fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:51:44.497821 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eee552-1b5d-4f1c-8dd7-137706d1e9fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:44.499162 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eee552-1b5d-4f1c-8dd7-137706d1e9fb None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:44.510679 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 614/1229] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:51:44.516483 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f080b8-1792-4639-98a0-df2875452b9c None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:44.526535 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:44.615692 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f080b8-1792-4639-98a0-df2875452b9c None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:44.648639 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b46ed-9de9-4e4a-be96-2341c9eef4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:44.649252 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b46ed-9de9-4e4a-be96-2341c9eef4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:44.649533 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b46ed-9de9-4e4a-be96-2341c9eef4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:44.649806 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b46ed-9de9-4e4a-be96-2341c9eef4f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:44.656226 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b46ed-9de9-4e4a-be96-2341c9eef4f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:44.657749 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b46ed-9de9-4e4a-be96-2341c9eef4f8 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:44.669902 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-691b46ed-9de9-4e4a-be96-2341c9eef4f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:51:44.669902 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:51:44.669902 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:51:44.669902 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:51:44.669902 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:51:44.669902 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:51:44.669902 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:51:44.669902 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:51:44.669902 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:51:44.669902 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:51:44.669902 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:51:44.669902 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:51:44.669902 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:51:44.669902 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:51:44.669902 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:51:44.670967 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:51:44.670967 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:51:44.670967 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:51:44.670967 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:51:44.670967 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:51:44.670967 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:51:44.670967 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:51:44.670967 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:51:44.670967 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:51:44.670967 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:51:44.670967 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:51:44.670967 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:51:44.670967 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:51:44.670967 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:51:44.670967 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:51:44.670967 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:51:44.670967 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:51:44.672107 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:51:44.672107 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:51:44.672107 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:51:44.672107 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:51:44.672107 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:51:44.672107 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:51:44.672107 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:51:44.672107 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:51:44.672107 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 616/1230] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:51:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:51:44.677099 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eee552-1b5d-4f1c-8dd7-137706d1e9fb None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:44.777611 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eee552-1b5d-4f1c-8dd7-137706d1e9fb None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:44.811782 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b31709-53d3-49b7-b556-e4607cadcd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:44.812376 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b31709-53d3-49b7-b556-e4607cadcd9f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:44.812725 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b31709-53d3-49b7-b556-e4607cadcd9f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:44.813036 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b31709-53d3-49b7-b556-e4607cadcd9f None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:44.813686 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b31709-53d3-49b7-b556-e4607cadcd9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:51:44.813990 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b31709-53d3-49b7-b556-e4607cadcd9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:44.816038 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b31709-53d3-49b7-b556-e4607cadcd9f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:44.827787 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 615/1231] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:51:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:51:44.834342 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b46ed-9de9-4e4a-be96-2341c9eef4f8 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:44.901699 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:44.934993 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b46ed-9de9-4e4a-be96-2341c9eef4f8 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:44.969410 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7effaa6-572d-44ce-a863-f098ba3e1c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:44.970052 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7effaa6-572d-44ce-a863-f098ba3e1c07 None admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:44.970331 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7effaa6-572d-44ce-a863-f098ba3e1c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:44.970645 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7effaa6-572d-44ce-a863-f098ba3e1c07 None admin] PATH_INFO: `/v3/projects/5100a8d316d243fb8f82e9044e1b3b06/users/018802a8d9c041119ad708430f885a89/roles/6e3e398ba29e436e8532a45d2fdf1321` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:44.984706 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7effaa6-572d-44ce-a863-f098ba3e1c07 None admin] RBAC: Authorizing `identity:create_grant(project_id=5100a8d316d243fb8f82e9044e1b3b06, user_id=018802a8d9c041119ad708430f885a89, role_id=6e3e398ba29e436e8532a45d2fdf1321)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:44.986175 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7effaa6-572d-44ce-a863-f098ba3e1c07 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:45.099671 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 617/1232] 10.222.0.27 () {58 vars in 1439 bytes} [Sun Sep 4 00:51:44 2022] PUT /identity/v3/projects/5100a8d316d243fb8f82e9044e1b3b06/users/018802a8d9c041119ad708430f885a89/roles/6e3e398ba29e436e8532a45d2fdf1321 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 00:51:46.968336 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f62a4-86ed-4a1f-aad5-8db171dd6ad7 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:46.968900 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f62a4-86ed-4a1f-aad5-8db171dd6ad7 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:46.968900 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f62a4-86ed-4a1f-aad5-8db171dd6ad7 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:46.970262 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 616/1233] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:51:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:51:46.976340 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b28dda-ad71-46dc-8890-b7bd187dcfbe None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:46.976766 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b28dda-ad71-46dc-8890-b7bd187dcfbe None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:46.976924 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b28dda-ad71-46dc-8890-b7bd187dcfbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:46.995465 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-95b28dda-ad71-46dc-8890-b7bd187dcfbe None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:51:46.996680 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-95b28dda-ad71-46dc-8890-b7bd187dcfbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:51:47.063116 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 618/1234] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:51:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:51:47.337166 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f62a4-86ed-4a1f-aad5-8db171dd6ad7 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:47.362431 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:47.437510 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f62a4-86ed-4a1f-aad5-8db171dd6ad7 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:47.471332 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee76447-999a-4ebe-ab8a-aed652505803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:47.471943 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee76447-999a-4ebe-ab8a-aed652505803 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:47.472248 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee76447-999a-4ebe-ab8a-aed652505803 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:47.472546 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee76447-999a-4ebe-ab8a-aed652505803 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:47.484253 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee76447-999a-4ebe-ab8a-aed652505803 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:51:47.487614 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-8ee76447-999a-4ebe-ab8a-aed652505803 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:51:47.487614 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:51:47.487614 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:51:47.487614 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:51:47.487614 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:51:47.487614 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:51:47.487614 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:51:47.487614 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:51:47.487614 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:51:47.487614 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:51:47.487614 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:51:47.487614 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:51:47.487614 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:51:47.487614 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:51:47.487614 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:51:47.487614 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:51:47.488751 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:51:47.488751 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:51:47.488751 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:51:47.488751 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:51:47.488751 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:51:47.488751 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:51:47.488751 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:51:47.488751 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:51:47.488751 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:51:47.488751 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:51:47.488751 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:51:47.488751 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:51:47.488751 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:51:47.488751 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:51:47.488751 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:51:47.488751 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:51:47.489601 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:51:47.489601 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:51:47.489601 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:51:47.489601 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:51:47.489601 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:51:47.489601 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:51:47.489601 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:51:47.489601 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:51:47.489601 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:51:47.489601 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:51:47.489601 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 04 00:51:47.489601 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:51:47.489601 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 617/1235] 10.222.0.27 () {56 vars in 1102 bytes} [Sun Sep 4 00:51:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:51:47.495078 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b28dda-ad71-46dc-8890-b7bd187dcfbe None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:47.542045 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:47.594774 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b28dda-ad71-46dc-8890-b7bd187dcfbe None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:47.628271 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde228e-57df-433e-b082-5a3e23bcc928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:47.628892 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde228e-57df-433e-b082-5a3e23bcc928 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:47.629174 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde228e-57df-433e-b082-5a3e23bcc928 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:47.629449 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde228e-57df-433e-b082-5a3e23bcc928 None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:47.630105 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcde228e-57df-433e-b082-5a3e23bcc928 None admin] RBAC: Adding query filter params (name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:51:47.630408 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcde228e-57df-433e-b082-5a3e23bcc928 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:47.631774 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcde228e-57df-433e-b082-5a3e23bcc928 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:47.639312 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 619/1236] 10.222.0.27 () {56 vars in 1103 bytes} [Sun Sep 4 00:51:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:51:47.645090 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee76447-999a-4ebe-ab8a-aed652505803 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:47.728001 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:47.743656 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee76447-999a-4ebe-ab8a-aed652505803 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:47.777277 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ecfa7-072d-4100-97db-77ce58406053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:47.777897 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ecfa7-072d-4100-97db-77ce58406053 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:47.778178 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ecfa7-072d-4100-97db-77ce58406053 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:47.778448 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ecfa7-072d-4100-97db-77ce58406053 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:47.788330 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433ecfa7-072d-4100-97db-77ce58406053 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:47.789820 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433ecfa7-072d-4100-97db-77ce58406053 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:47.804321 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-433ecfa7-072d-4100-97db-77ce58406053 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:51:47.804321 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:51:47.804321 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:51:47.804321 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:51:47.804321 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:51:47.804321 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:51:47.804321 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:51:47.804321 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:51:47.804321 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:51:47.804321 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:51:47.804321 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:51:47.804321 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:51:47.804321 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:51:47.804321 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:51:47.804321 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:51:47.804321 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:51:47.805584 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:51:47.805584 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:51:47.805584 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:51:47.805584 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:51:47.805584 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:51:47.805584 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:51:47.805584 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:51:47.805584 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:51:47.805584 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:51:47.805584 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:51:47.805584 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:51:47.805584 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:51:47.805584 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:51:47.805584 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:51:47.805584 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:51:47.805584 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:51:47.805584 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:51:47.806764 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:51:47.806764 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:51:47.806764 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:51:47.806764 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 618/1237] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:51:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:51:47.811355 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde228e-57df-433e-b082-5a3e23bcc928 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:47.911083 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde228e-57df-433e-b082-5a3e23bcc928 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:47.919792 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:47.943924 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ede3d4-283b-4bb7-9ab0-0747472a1ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:47.944537 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ede3d4-283b-4bb7-9ab0-0747472a1ceb None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:47.944816 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ede3d4-283b-4bb7-9ab0-0747472a1ceb None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:47.945085 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ede3d4-283b-4bb7-9ab0-0747472a1ceb None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:47.945713 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ede3d4-283b-4bb7-9ab0-0747472a1ceb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:51:47.946011 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ede3d4-283b-4bb7-9ab0-0747472a1ceb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:47.947370 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ede3d4-283b-4bb7-9ab0-0747472a1ceb None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:47.958971 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 620/1238] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:51:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:51:47.964662 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ecfa7-072d-4100-97db-77ce58406053 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:48.066018 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ecfa7-072d-4100-97db-77ce58406053 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:48.098822 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48304a2d-c5be-43b6-8cc4-6a032a884c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:48.099440 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48304a2d-c5be-43b6-8cc4-6a032a884c40 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:48.099720 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48304a2d-c5be-43b6-8cc4-6a032a884c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:48.099992 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48304a2d-c5be-43b6-8cc4-6a032a884c40 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:48.102644 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:48.107450 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48304a2d-c5be-43b6-8cc4-6a032a884c40 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:48.108926 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48304a2d-c5be-43b6-8cc4-6a032a884c40 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:48.120649 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-48304a2d-c5be-43b6-8cc4-6a032a884c40 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 04 00:51:48.120649 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:51:48.120649 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:51:48.120649 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:51:48.120649 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:51:48.120649 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:51:48.120649 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:51:48.120649 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:51:48.120649 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:51:48.120649 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:51:48.120649 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:51:48.120649 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:51:48.120649 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:51:48.120649 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:51:48.120649 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:51:48.120649 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:51:48.121782 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:51:48.121782 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:51:48.121782 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:51:48.121782 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:51:48.121782 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:51:48.121782 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:51:48.121782 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:51:48.121782 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:51:48.121782 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:51:48.121782 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:51:48.121782 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:51:48.121782 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:51:48.121782 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:51:48.121782 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:51:48.121782 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:51:48.121782 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:51:48.121782 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:51:48.121782 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:51:48.122831 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:51:48.122831 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:51:48.122831 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:51:48.122831 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:51:48.122831 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:51:48.122831 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:51:48.122831 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:51:48.122831 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:51:48.122831 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 04 00:51:48.122831 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:51:48.122831 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 619/1239] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:51:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:51:48.127886 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ede3d4-283b-4bb7-9ab0-0747472a1ceb None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:48.228053 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ede3d4-283b-4bb7-9ab0-0747472a1ceb None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:48.262430 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a05ab-7e6f-4fcb-b429-01eb87974aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:48.263037 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a05ab-7e6f-4fcb-b429-01eb87974aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:48.263319 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a05ab-7e6f-4fcb-b429-01eb87974aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:48.263595 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a05ab-7e6f-4fcb-b429-01eb87974aa8 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:48.264312 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092a05ab-7e6f-4fcb-b429-01eb87974aa8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:51:48.264646 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092a05ab-7e6f-4fcb-b429-01eb87974aa8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:48.266084 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092a05ab-7e6f-4fcb-b429-01eb87974aa8 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:48.284578 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 621/1240] 10.222.0.27 () {56 vars in 1143 bytes} [Sun Sep 4 00:51:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:51:48.290344 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48304a2d-c5be-43b6-8cc4-6a032a884c40 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:48.391764 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48304a2d-c5be-43b6-8cc4-6a032a884c40 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:48.425881 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9ac0e-351f-493e-a5ea-02eb4f8a2d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:48.426469 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9ac0e-351f-493e-a5ea-02eb4f8a2d33 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:48.426747 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9ac0e-351f-493e-a5ea-02eb4f8a2d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:48.427079 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9ac0e-351f-493e-a5ea-02eb4f8a2d33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:48.486487 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:48.493280 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f9ac0e-351f-493e-a5ea-02eb4f8a2d33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:48.494746 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f9ac0e-351f-493e-a5ea-02eb4f8a2d33 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:48.592312 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 620/1241] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:51:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 04 00:51:48.598771 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a05ab-7e6f-4fcb-b429-01eb87974aa8 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:48.625582 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:48.698644 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a05ab-7e6f-4fcb-b429-01eb87974aa8 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:48.731803 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5eb3fe-5be5-4e71-8d46-01ea30c7764c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:48.732425 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5eb3fe-5be5-4e71-8d46-01ea30c7764c None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:48.732737 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5eb3fe-5be5-4e71-8d46-01ea30c7764c None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:48.733013 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5eb3fe-5be5-4e71-8d46-01ea30c7764c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:48.742681 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5eb3fe-5be5-4e71-8d46-01ea30c7764c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:48.744134 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5eb3fe-5be5-4e71-8d46-01ea30c7764c None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:48.758884 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-3e5eb3fe-5be5-4e71-8d46-01ea30c7764c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:51:48.758884 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:51:48.758884 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:51:48.758884 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:51:48.758884 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:51:48.758884 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:51:48.758884 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:51:48.758884 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:51:48.758884 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:51:48.758884 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:51:48.758884 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:51:48.758884 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:51:48.758884 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:51:48.758884 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:51:48.758884 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:51:48.758884 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:51:48.760001 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:51:48.760001 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:51:48.760001 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:51:48.760001 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:51:48.760001 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:51:48.760001 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:51:48.760001 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:51:48.760001 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:51:48.760001 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:51:48.760001 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:51:48.760001 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:51:48.760001 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:51:48.760001 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:51:48.760001 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:51:48.760001 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:51:48.760001 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:51:48.760001 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:51:48.761227 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:51:48.761227 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:51:48.761227 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:51:48.761227 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 622/1242] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:51:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:51:48.765848 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9ac0e-351f-493e-a5ea-02eb4f8a2d33 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:48.858248 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:48.864590 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9ac0e-351f-493e-a5ea-02eb4f8a2d33 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:48.898925 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40df4764-bb48-4e36-92e2-3bd66bf8c99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:48.899514 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df4764-bb48-4e36-92e2-3bd66bf8c99e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:48.899794 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df4764-bb48-4e36-92e2-3bd66bf8c99e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:48.900068 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df4764-bb48-4e36-92e2-3bd66bf8c99e None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:48.900722 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40df4764-bb48-4e36-92e2-3bd66bf8c99e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:51:48.901023 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40df4764-bb48-4e36-92e2-3bd66bf8c99e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:48.902383 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40df4764-bb48-4e36-92e2-3bd66bf8c99e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:48.914602 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 621/1243] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:51:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:51:48.920698 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5eb3fe-5be5-4e71-8d46-01ea30c7764c None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:49.002263 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:49.022987 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5eb3fe-5be5-4e71-8d46-01ea30c7764c None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:49.056705 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b61be-2dc5-47db-a2ce-7dff3b0fb245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:49.057316 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b61be-2dc5-47db-a2ce-7dff3b0fb245 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:49.057594 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b61be-2dc5-47db-a2ce-7dff3b0fb245 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:49.057864 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b61be-2dc5-47db-a2ce-7dff3b0fb245 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:49.065232 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05b61be-2dc5-47db-a2ce-7dff3b0fb245 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:49.066603 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05b61be-2dc5-47db-a2ce-7dff3b0fb245 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:49.077497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-c05b61be-2dc5-47db-a2ce-7dff3b0fb245 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:51:49.077497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:51:49.077497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:51:49.077497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:51:49.077497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:51:49.077497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:51:49.077497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:51:49.077497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:51:49.077497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:51:49.077497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:51:49.077497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:51:49.077497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:51:49.077497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:51:49.077497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:51:49.077497 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:51:49.078642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:51:49.078642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:51:49.078642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:51:49.078642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:51:49.078642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:51:49.078642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:51:49.078642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:51:49.078642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:51:49.078642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:51:49.078642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:51:49.078642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:51:49.078642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:51:49.078642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:51:49.078642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:51:49.078642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:51:49.078642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:51:49.078642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:51:49.079698 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:51:49.079698 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:51:49.079698 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:51:49.079698 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:51:49.079698 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:51:49.079698 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:51:49.079698 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:51:49.079698 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:51:49.079698 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 623/1244] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:51:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:51:49.085237 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40df4764-bb48-4e36-92e2-3bd66bf8c99e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:49.184095 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40df4764-bb48-4e36-92e2-3bd66bf8c99e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:49.216914 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b89e71d-c4a1-477f-9dd8-e791c1f0d2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:49.217510 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89e71d-c4a1-477f-9dd8-e791c1f0d2b8 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:49.217793 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89e71d-c4a1-477f-9dd8-e791c1f0d2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:49.218072 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89e71d-c4a1-477f-9dd8-e791c1f0d2b8 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:49.218753 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b89e71d-c4a1-477f-9dd8-e791c1f0d2b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:51:49.219055 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b89e71d-c4a1-477f-9dd8-e791c1f0d2b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:49.221065 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b89e71d-c4a1-477f-9dd8-e791c1f0d2b8 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:49.233713 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 622/1245] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:51:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:51:49.240039 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b61be-2dc5-47db-a2ce-7dff3b0fb245 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:49.340050 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b61be-2dc5-47db-a2ce-7dff3b0fb245 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:49.372811 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5938bc7-09f6-42c0-8c7f-75efe7597a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:49.373416 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5938bc7-09f6-42c0-8c7f-75efe7597a39 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:49.373699 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5938bc7-09f6-42c0-8c7f-75efe7597a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:49.373972 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5938bc7-09f6-42c0-8c7f-75efe7597a39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:49.374688 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5938bc7-09f6-42c0-8c7f-75efe7597a39 None admin] RBAC: Adding query filter params (role.id=6e3e398ba29e436e8532a45d2fdf1321, scope.project.id=5100a8d316d243fb8f82e9044e1b3b06, user.id=018802a8d9c041119ad708430f885a89) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:51:49.374996 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5938bc7-09f6-42c0-8c7f-75efe7597a39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:49.376433 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5938bc7-09f6-42c0-8c7f-75efe7597a39 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:49.382855 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 624/1246] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:51:49 2022] GET /identity/v3/role_assignments?user.id=018802a8d9c041119ad708430f885a89&scope.project.id=5100a8d316d243fb8f82e9044e1b3b06&role.id=6e3e398ba29e436e8532a45d2fdf1321 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:51:51.322097 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43974fcf-ff32-4c66-8170-b6f969ba9bcd None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:51.322644 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43974fcf-ff32-4c66-8170-b6f969ba9bcd None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:51.322644 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43974fcf-ff32-4c66-8170-b6f969ba9bcd None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:51.324018 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 623/1247] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:51:51.329139 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f3803-94fa-4e1f-b1b8-49fe71dfc74c None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:51.329572 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f3803-94fa-4e1f-b1b8-49fe71dfc74c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:51.330269 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f3803-94fa-4e1f-b1b8-49fe71dfc74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:51.336424 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:51.349091 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-943f3803-94fa-4e1f-b1b8-49fe71dfc74c None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:51:51.350274 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-943f3803-94fa-4e1f-b1b8-49fe71dfc74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:51:51.414595 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 625/1248] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:51:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:51:51.685404 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43974fcf-ff32-4c66-8170-b6f969ba9bcd None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:51.705320 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:51.788097 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43974fcf-ff32-4c66-8170-b6f969ba9bcd None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:51.821657 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce3e04-6068-4510-8d58-e9374176926a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:51.822299 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce3e04-6068-4510-8d58-e9374176926a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:51.822578 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce3e04-6068-4510-8d58-e9374176926a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:51.822855 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce3e04-6068-4510-8d58-e9374176926a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:51.833369 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce3e04-6068-4510-8d58-e9374176926a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:51:51.838122 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-47ce3e04-6068-4510-8d58-e9374176926a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:51:51.838122 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:51:51.838122 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:51:51.838122 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:51:51.838122 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:51:51.838122 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:51:51.838122 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:51:51.838122 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:51:51.838122 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:51:51.838122 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:51:51.838122 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:51:51.838122 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:51:51.838122 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:51:51.838122 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:51:51.838122 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:51:51.838122 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:51:51.839167 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:51:51.839167 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:51:51.839167 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:51:51.839167 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:51:51.839167 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:51:51.839167 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:51:51.839167 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:51:51.839167 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:51:51.839167 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:51:51.839167 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:51:51.839167 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:51:51.839167 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:51:51.839167 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:51:51.839167 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:51:51.839167 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:51:51.839167 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:51:51.839999 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:51:51.839999 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:51:51.839999 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:51:51.839999 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:51:51.839999 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:51:51.839999 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:51:51.839999 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:51:51.839999 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:51:51.839999 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:51:51.839999 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:51:51.839999 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:51:51.839999 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:51:51.839999 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 624/1249] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:51:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:51:51.845430 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f3803-94fa-4e1f-b1b8-49fe71dfc74c None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:51.944572 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f3803-94fa-4e1f-b1b8-49fe71dfc74c None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:51.974508 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:51.979279 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e54de-55d7-4a3a-a83a-c63ef2933f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:51.979878 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e54de-55d7-4a3a-a83a-c63ef2933f9a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:51.980156 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e54de-55d7-4a3a-a83a-c63ef2933f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:51.980426 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e54de-55d7-4a3a-a83a-c63ef2933f9a None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:51.981109 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00e54de-55d7-4a3a-a83a-c63ef2933f9a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:51:51.981406 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00e54de-55d7-4a3a-a83a-c63ef2933f9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:51.982769 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00e54de-55d7-4a3a-a83a-c63ef2933f9a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:51.991311 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 626/1250] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:51:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:51:51.997287 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce3e04-6068-4510-8d58-e9374176926a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:52.074862 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:52.097287 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce3e04-6068-4510-8d58-e9374176926a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:52.130926 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e9a5f-c3cc-4c5f-81a3-c32b5b822e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:52.131532 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e9a5f-c3cc-4c5f-81a3-c32b5b822e9e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:52.131808 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e9a5f-c3cc-4c5f-81a3-c32b5b822e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:52.132078 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e9a5f-c3cc-4c5f-81a3-c32b5b822e9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:52.141745 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e9a5f-c3cc-4c5f-81a3-c32b5b822e9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:52.143961 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e9a5f-c3cc-4c5f-81a3-c32b5b822e9e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:52.157182 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-237e9a5f-c3cc-4c5f-81a3-c32b5b822e9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:51:52.157182 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:51:52.157182 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:51:52.157182 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:51:52.157182 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:51:52.157182 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:51:52.157182 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:51:52.157182 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:51:52.157182 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:51:52.157182 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:51:52.157182 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:51:52.157182 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:51:52.157182 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:51:52.157182 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:51:52.157182 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:51:52.157182 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:51:52.163711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:51:52.163711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:51:52.163711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:51:52.163711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:51:52.163711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:51:52.163711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:51:52.163711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:51:52.163711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:51:52.163711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:51:52.163711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:51:52.163711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:51:52.163711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:51:52.163711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:51:52.163711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:51:52.163711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:51:52.163711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:51:52.163711 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:51:52.164792 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:51:52.164792 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:51:52.164792 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:51:52.164792 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 625/1251] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:51:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:51:52.165086 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e54de-55d7-4a3a-a83a-c63ef2933f9a None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:52.265277 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e54de-55d7-4a3a-a83a-c63ef2933f9a None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:52.299989 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce0779-d9b6-4fdd-9ba9-5629345cf7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:52.300630 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce0779-d9b6-4fdd-9ba9-5629345cf7d7 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:52.300914 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce0779-d9b6-4fdd-9ba9-5629345cf7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:52.301189 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce0779-d9b6-4fdd-9ba9-5629345cf7d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:52.301822 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce0779-d9b6-4fdd-9ba9-5629345cf7d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:51:52.302124 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce0779-d9b6-4fdd-9ba9-5629345cf7d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:52.303500 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce0779-d9b6-4fdd-9ba9-5629345cf7d7 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:52.315871 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 627/1252] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:51:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:51:52.321707 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e9a5f-c3cc-4c5f-81a3-c32b5b822e9e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:52.422878 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e9a5f-c3cc-4c5f-81a3-c32b5b822e9e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:52.449436 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:52.457111 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5e976b-894e-4948-95c5-a204a84d8194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:52.457722 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e976b-894e-4948-95c5-a204a84d8194 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:52.457999 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e976b-894e-4948-95c5-a204a84d8194 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:52.458313 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e976b-894e-4948-95c5-a204a84d8194 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:52.465403 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5e976b-894e-4948-95c5-a204a84d8194 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:52.466831 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5e976b-894e-4948-95c5-a204a84d8194 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:52.478299 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-cd5e976b-894e-4948-95c5-a204a84d8194 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 04 00:51:52.478299 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:51:52.478299 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:51:52.478299 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:51:52.478299 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:51:52.478299 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:51:52.478299 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:51:52.478299 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:51:52.478299 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:51:52.478299 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:51:52.478299 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:51:52.478299 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:51:52.478299 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:51:52.478299 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:51:52.478299 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:51:52.478299 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:51:52.481697 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:51:52.481697 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:51:52.481697 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:51:52.481697 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:51:52.481697 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:51:52.481697 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:51:52.481697 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:51:52.481697 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:51:52.481697 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:51:52.481697 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:51:52.481697 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:51:52.481697 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:51:52.481697 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:51:52.481697 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:51:52.481697 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:51:52.481697 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:51:52.481697 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:51:52.481697 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:51:52.482887 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:51:52.482887 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:51:52.482887 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:51:52.482887 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:51:52.482887 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:51:52.482887 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:51:52.482887 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:51:52.482887 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:51:52.482887 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 04 00:51:52.482887 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:51:52.482887 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 626/1253] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:51:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:51:52.485455 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce0779-d9b6-4fdd-9ba9-5629345cf7d7 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:52.541674 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:52.583849 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce0779-d9b6-4fdd-9ba9-5629345cf7d7 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:52.617409 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f4fc3-050b-4374-83d5-181fa75e0979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:52.618005 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f4fc3-050b-4374-83d5-181fa75e0979 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:52.618326 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f4fc3-050b-4374-83d5-181fa75e0979 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:52.618598 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f4fc3-050b-4374-83d5-181fa75e0979 None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:52.619309 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f4fc3-050b-4374-83d5-181fa75e0979 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:51:52.619613 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f4fc3-050b-4374-83d5-181fa75e0979 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:52.621068 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f4fc3-050b-4374-83d5-181fa75e0979 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:52.637900 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 628/1254] 10.222.0.27 () {56 vars in 1143 bytes} [Sun Sep 4 00:51:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 00:51:52.644095 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5e976b-894e-4948-95c5-a204a84d8194 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:52.746481 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5e976b-894e-4948-95c5-a204a84d8194 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:52.779653 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d831df4-7d30-4992-95e9-70ec04695f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:52.780245 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d831df4-7d30-4992-95e9-70ec04695f83 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:52.780551 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d831df4-7d30-4992-95e9-70ec04695f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:52.780826 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d831df4-7d30-4992-95e9-70ec04695f83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:52.835714 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:52.848534 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d831df4-7d30-4992-95e9-70ec04695f83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:52.850041 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d831df4-7d30-4992-95e9-70ec04695f83 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:52.946711 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 627/1255] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:51:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 04 00:51:52.952771 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f4fc3-050b-4374-83d5-181fa75e0979 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:53.051905 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f4fc3-050b-4374-83d5-181fa75e0979 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:53.065528 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:53.086630 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5115e9-99eb-4387-992d-ba0c2b1e2b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:53.087251 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5115e9-99eb-4387-992d-ba0c2b1e2b59 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:53.087527 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5115e9-99eb-4387-992d-ba0c2b1e2b59 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:53.087797 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5115e9-99eb-4387-992d-ba0c2b1e2b59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:53.098150 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5115e9-99eb-4387-992d-ba0c2b1e2b59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:53.099580 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5115e9-99eb-4387-992d-ba0c2b1e2b59 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:53.113651 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-0f5115e9-99eb-4387-992d-ba0c2b1e2b59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:51:53.113651 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:51:53.113651 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:51:53.113651 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:51:53.113651 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:51:53.113651 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:51:53.113651 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:51:53.113651 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:51:53.113651 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:51:53.113651 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:51:53.113651 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:51:53.113651 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:51:53.113651 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:51:53.113651 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:51:53.113651 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:51:53.113651 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:51:53.114789 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:51:53.114789 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:51:53.114789 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:51:53.114789 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:51:53.114789 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:51:53.114789 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:51:53.114789 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:51:53.114789 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:51:53.114789 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:51:53.114789 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:51:53.114789 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:51:53.114789 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:51:53.114789 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:51:53.114789 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:51:53.114789 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:51:53.114789 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:51:53.114789 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:51:53.115907 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:51:53.115907 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:51:53.115907 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:51:53.115907 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 629/1256] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:51:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:51:53.120412 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d831df4-7d30-4992-95e9-70ec04695f83 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:53.207429 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:53.218870 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d831df4-7d30-4992-95e9-70ec04695f83 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:53.253056 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84941d3-78de-48f0-b320-c575743b6e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:53.253663 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84941d3-78de-48f0-b320-c575743b6e00 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:53.253942 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84941d3-78de-48f0-b320-c575743b6e00 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:53.254290 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84941d3-78de-48f0-b320-c575743b6e00 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:53.254928 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84941d3-78de-48f0-b320-c575743b6e00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:51:53.255258 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84941d3-78de-48f0-b320-c575743b6e00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:53.256712 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84941d3-78de-48f0-b320-c575743b6e00 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:53.268493 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 628/1257] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:51:53.274200 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5115e9-99eb-4387-992d-ba0c2b1e2b59 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:53.371456 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5115e9-99eb-4387-992d-ba0c2b1e2b59 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:53.404321 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f4332-2363-4900-ba0d-bf136ad23845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:53.404954 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f4332-2363-4900-ba0d-bf136ad23845 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:53.405231 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f4332-2363-4900-ba0d-bf136ad23845 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:53.405501 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f4332-2363-4900-ba0d-bf136ad23845 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:53.412433 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462f4332-2363-4900-ba0d-bf136ad23845 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:53.413931 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462f4332-2363-4900-ba0d-bf136ad23845 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:53.424880 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-462f4332-2363-4900-ba0d-bf136ad23845 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:51:53.424880 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:51:53.424880 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:51:53.424880 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:51:53.424880 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:51:53.424880 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:51:53.424880 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:51:53.424880 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:51:53.424880 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:51:53.424880 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:51:53.424880 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:51:53.424880 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:51:53.424880 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:51:53.424880 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:51:53.424880 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:51:53.425952 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:51:53.425952 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:51:53.425952 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:51:53.425952 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:51:53.425952 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:51:53.425952 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:51:53.425952 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:51:53.425952 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:51:53.425952 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:51:53.425952 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:51:53.425952 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:51:53.425952 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:51:53.425952 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:51:53.425952 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:51:53.425952 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:51:53.425952 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:51:53.425952 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:51:53.427147 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:51:53.427147 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:51:53.427147 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:51:53.427147 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:51:53.427147 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:51:53.427147 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:51:53.427147 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:51:53.427147 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:51:53.428601 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 630/1258] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:51:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:51:53.431792 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84941d3-78de-48f0-b320-c575743b6e00 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:53.531783 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84941d3-78de-48f0-b320-c575743b6e00 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:53.565502 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66159ccd-942f-42a9-aff8-4f0300bbee78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:53.566098 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66159ccd-942f-42a9-aff8-4f0300bbee78 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:53.566377 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66159ccd-942f-42a9-aff8-4f0300bbee78 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:53.566648 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66159ccd-942f-42a9-aff8-4f0300bbee78 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:53.567329 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66159ccd-942f-42a9-aff8-4f0300bbee78 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:51:53.567631 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66159ccd-942f-42a9-aff8-4f0300bbee78 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:53.569075 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66159ccd-942f-42a9-aff8-4f0300bbee78 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:53.581315 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 629/1259] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:51:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:51:53.587312 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f4332-2363-4900-ba0d-bf136ad23845 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:53.612946 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:53.685894 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f4332-2363-4900-ba0d-bf136ad23845 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:53.722288 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43b463-c96c-4d69-8061-6a2b3ed11e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:53.722888 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43b463-c96c-4d69-8061-6a2b3ed11e83 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:53.723164 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43b463-c96c-4d69-8061-6a2b3ed11e83 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:53.723437 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43b463-c96c-4d69-8061-6a2b3ed11e83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:53.724154 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c43b463-c96c-4d69-8061-6a2b3ed11e83 None admin] RBAC: Adding query filter params (role.id=c8a59d34ee2f4cefa6940cac778260c7, scope.project.id=5100a8d316d243fb8f82e9044e1b3b06, user.id=018802a8d9c041119ad708430f885a89) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:51:53.724481 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c43b463-c96c-4d69-8061-6a2b3ed11e83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:53.725901 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c43b463-c96c-4d69-8061-6a2b3ed11e83 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:53.731120 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 631/1260] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:51:53 2022] GET /identity/v3/role_assignments?user.id=018802a8d9c041119ad708430f885a89&scope.project.id=5100a8d316d243fb8f82e9044e1b3b06&role.id=c8a59d34ee2f4cefa6940cac778260c7 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:51:55.596683 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc7fb2-02ba-408c-a2a4-7ee82467fdf0 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:55.597245 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc7fb2-02ba-408c-a2a4-7ee82467fdf0 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:55.597344 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc7fb2-02ba-408c-a2a4-7ee82467fdf0 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:55.598682 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 630/1261] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:51:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:51:55.603762 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef564cbd-ccbf-45b6-b5f0-f033c9cb579c None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:55.604153 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef564cbd-ccbf-45b6-b5f0-f033c9cb579c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:55.604313 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef564cbd-ccbf-45b6-b5f0-f033c9cb579c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:55.623196 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-ef564cbd-ccbf-45b6-b5f0-f033c9cb579c None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:51:55.626985 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ef564cbd-ccbf-45b6-b5f0-f033c9cb579c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:51:55.690038 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 632/1262] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:51:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:51:55.955332 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc7fb2-02ba-408c-a2a4-7ee82467fdf0 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:55.966342 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:56.057811 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc7fb2-02ba-408c-a2a4-7ee82467fdf0 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:56.091814 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1072c75c-8fef-409c-a8b5-a92bfd124265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:56.092440 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1072c75c-8fef-409c-a8b5-a92bfd124265 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:56.092752 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1072c75c-8fef-409c-a8b5-a92bfd124265 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:56.093031 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1072c75c-8fef-409c-a8b5-a92bfd124265 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:56.103594 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1072c75c-8fef-409c-a8b5-a92bfd124265 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:51:56.108426 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-1072c75c-8fef-409c-a8b5-a92bfd124265 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:51:56.108426 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:51:56.108426 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:51:56.108426 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:51:56.108426 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:51:56.108426 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:51:56.108426 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:51:56.108426 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:51:56.108426 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:51:56.108426 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:51:56.108426 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:51:56.108426 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:51:56.108426 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:51:56.108426 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:51:56.108426 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:51:56.108426 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:51:56.109594 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:51:56.109594 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:51:56.109594 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:51:56.109594 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:51:56.109594 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:51:56.109594 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:51:56.109594 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:51:56.109594 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:51:56.109594 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:51:56.109594 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:51:56.109594 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:51:56.109594 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:51:56.109594 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:51:56.109594 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:51:56.109594 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:51:56.109594 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:51:56.110459 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:51:56.110459 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:51:56.110459 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:51:56.110459 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:51:56.110459 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:51:56.110459 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:51:56.110459 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:51:56.110459 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:51:56.110459 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:51:56.110459 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:51:56.110459 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:51:56.110459 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:51:56.110459 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 631/1263] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:51:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:51:56.115739 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef564cbd-ccbf-45b6-b5f0-f033c9cb579c None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:56.134393 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:56.216610 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef564cbd-ccbf-45b6-b5f0-f033c9cb579c None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:56.252942 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1367f30d-4361-42ab-8dee-78d4aafa203a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:56.253548 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367f30d-4361-42ab-8dee-78d4aafa203a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:56.253830 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367f30d-4361-42ab-8dee-78d4aafa203a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:56.254107 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367f30d-4361-42ab-8dee-78d4aafa203a None admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:56.254766 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1367f30d-4361-42ab-8dee-78d4aafa203a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:51:56.255069 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1367f30d-4361-42ab-8dee-78d4aafa203a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:56.256435 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1367f30d-4361-42ab-8dee-78d4aafa203a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:56.264336 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 633/1264] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:51:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:51:56.270846 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1072c75c-8fef-409c-a8b5-a92bfd124265 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:56.340721 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:56.372944 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1072c75c-8fef-409c-a8b5-a92bfd124265 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:56.406489 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590eafe5-38da-4372-ab40-249d769c4589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:56.407107 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590eafe5-38da-4372-ab40-249d769c4589 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:56.407388 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590eafe5-38da-4372-ab40-249d769c4589 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:56.407733 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590eafe5-38da-4372-ab40-249d769c4589 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:56.418178 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590eafe5-38da-4372-ab40-249d769c4589 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:56.419601 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590eafe5-38da-4372-ab40-249d769c4589 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:56.433569 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-590eafe5-38da-4372-ab40-249d769c4589 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:51:56.433569 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:51:56.433569 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:51:56.433569 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:51:56.433569 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:51:56.433569 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:51:56.433569 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:51:56.433569 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:51:56.433569 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:51:56.433569 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:51:56.433569 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:51:56.433569 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:51:56.433569 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:51:56.433569 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:51:56.433569 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:51:56.433569 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:51:56.434733 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:51:56.434733 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:51:56.434733 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:51:56.434733 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:51:56.434733 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:51:56.434733 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:51:56.434733 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:51:56.434733 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:51:56.434733 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:51:56.434733 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:51:56.434733 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:51:56.434733 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:51:56.434733 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:51:56.434733 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:51:56.434733 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:51:56.434733 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:51:56.434733 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:51:56.435715 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:51:56.435715 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:51:56.435715 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:51:56.435715 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 632/1265] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:51:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:51:56.441456 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1367f30d-4361-42ab-8dee-78d4aafa203a None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:56.523557 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:56.542038 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1367f30d-4361-42ab-8dee-78d4aafa203a None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:56.575499 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae0362c-55eb-46f4-9579-c859f550695f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:56.576090 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0362c-55eb-46f4-9579-c859f550695f None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:56.576367 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0362c-55eb-46f4-9579-c859f550695f None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:56.576665 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0362c-55eb-46f4-9579-c859f550695f None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:56.577294 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae0362c-55eb-46f4-9579-c859f550695f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:51:56.577594 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae0362c-55eb-46f4-9579-c859f550695f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:56.578951 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae0362c-55eb-46f4-9579-c859f550695f None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:56.590038 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 634/1266] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:51:56.595871 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590eafe5-38da-4372-ab40-249d769c4589 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:56.698680 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590eafe5-38da-4372-ab40-249d769c4589 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:56.718439 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:56.732680 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084cc59d-89f7-4db3-a40d-9cbcbc7d67a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:56.733296 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084cc59d-89f7-4db3-a40d-9cbcbc7d67a3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:56.733577 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084cc59d-89f7-4db3-a40d-9cbcbc7d67a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:56.733851 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084cc59d-89f7-4db3-a40d-9cbcbc7d67a3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:56.742304 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084cc59d-89f7-4db3-a40d-9cbcbc7d67a3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:56.743798 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084cc59d-89f7-4db3-a40d-9cbcbc7d67a3 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:56.755381 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-084cc59d-89f7-4db3-a40d-9cbcbc7d67a3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 04 00:51:56.755381 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:51:56.755381 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:51:56.755381 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:51:56.755381 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:51:56.755381 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:51:56.755381 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:51:56.755381 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:51:56.755381 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:51:56.755381 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:51:56.755381 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:51:56.755381 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:51:56.755381 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:51:56.755381 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:51:56.755381 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:51:56.755381 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:51:56.757176 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:51:56.757176 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:51:56.757176 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:51:56.757176 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:51:56.757176 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:51:56.757176 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:51:56.757176 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:51:56.757176 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:51:56.757176 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:51:56.757176 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:51:56.757176 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:51:56.757176 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:51:56.757176 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:51:56.757176 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:51:56.757176 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:51:56.757176 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:51:56.757176 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:51:56.757176 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:51:56.758616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:51:56.758616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:51:56.758616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:51:56.758616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:51:56.758616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:51:56.758616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:51:56.758616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:51:56.758616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:51:56.758616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 04 00:51:56.758616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:51:56.758616 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 633/1267] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:51:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:51:56.762292 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae0362c-55eb-46f4-9579-c859f550695f None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:56.867417 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae0362c-55eb-46f4-9579-c859f550695f None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:56.899608 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ac756-e22b-4848-8bf6-4679e1751dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:56.900202 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ac756-e22b-4848-8bf6-4679e1751dde None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:56.900997 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ac756-e22b-4848-8bf6-4679e1751dde None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:56.900997 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ac756-e22b-4848-8bf6-4679e1751dde None admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:56.901473 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ac756-e22b-4848-8bf6-4679e1751dde None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:51:56.901770 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ac756-e22b-4848-8bf6-4679e1751dde None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:56.903245 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ac756-e22b-4848-8bf6-4679e1751dde None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:56.913016 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:56.921143 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 635/1268] 10.222.0.27 () {56 vars in 1143 bytes} [Sun Sep 4 00:51:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:51:56.927501 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084cc59d-89f7-4db3-a40d-9cbcbc7d67a3 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:57.026260 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084cc59d-89f7-4db3-a40d-9cbcbc7d67a3 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:57.060770 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0cd28-73ae-4050-b87d-1b4c5e494ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:57.061381 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0cd28-73ae-4050-b87d-1b4c5e494ff3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:57.061659 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0cd28-73ae-4050-b87d-1b4c5e494ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:57.061926 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0cd28-73ae-4050-b87d-1b4c5e494ff3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:57.072125 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0cd28-73ae-4050-b87d-1b4c5e494ff3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:57.073587 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0cd28-73ae-4050-b87d-1b4c5e494ff3 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:57.087158 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-d0d0cd28-73ae-4050-b87d-1b4c5e494ff3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:51:57.087158 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:51:57.087158 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:51:57.087158 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:51:57.087158 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:51:57.087158 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:51:57.087158 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:51:57.087158 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:51:57.087158 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:51:57.087158 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:51:57.087158 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:51:57.087158 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:51:57.087158 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:51:57.087158 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:51:57.087158 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:51:57.087158 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:51:57.092438 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:51:57.092438 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:51:57.092438 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:51:57.092438 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:51:57.092438 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:51:57.092438 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:51:57.092438 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:51:57.092438 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:51:57.092438 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:51:57.092438 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:51:57.092438 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:51:57.092438 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:51:57.092438 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:51:57.092438 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:51:57.092438 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:51:57.092438 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:51:57.092438 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:51:57.093588 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:51:57.093588 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:51:57.093588 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:51:57.093588 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 634/1269] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:51:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:51:57.094131 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ac756-e22b-4848-8bf6-4679e1751dde None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:57.191290 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ac756-e22b-4848-8bf6-4679e1751dde None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:57.224786 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0996b3-b552-40da-91b8-c52746db7fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:57.225407 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0996b3-b552-40da-91b8-c52746db7fda None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:57.225692 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0996b3-b552-40da-91b8-c52746db7fda None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:57.225970 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0996b3-b552-40da-91b8-c52746db7fda None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:57.226620 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0996b3-b552-40da-91b8-c52746db7fda None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:51:57.226938 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0996b3-b552-40da-91b8-c52746db7fda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:57.228419 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0996b3-b552-40da-91b8-c52746db7fda None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:57.240857 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 636/1270] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:51:57.246515 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0cd28-73ae-4050-b87d-1b4c5e494ff3 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:57.277821 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:57.345763 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0cd28-73ae-4050-b87d-1b4c5e494ff3 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:57.379585 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a85d39-849f-44a4-90be-26bb86e9904e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:57.380200 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a85d39-849f-44a4-90be-26bb86e9904e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:57.380505 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a85d39-849f-44a4-90be-26bb86e9904e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:57.380851 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a85d39-849f-44a4-90be-26bb86e9904e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:57.387318 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a85d39-849f-44a4-90be-26bb86e9904e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:57.388804 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a85d39-849f-44a4-90be-26bb86e9904e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:57.400416 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-e7a85d39-849f-44a4-90be-26bb86e9904e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:51:57.400416 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:51:57.400416 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:51:57.400416 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:51:57.400416 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:51:57.400416 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:51:57.400416 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:51:57.400416 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:51:57.400416 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:51:57.400416 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:51:57.400416 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:51:57.400416 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:51:57.400416 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:51:57.400416 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:51:57.400416 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:51:57.401583 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:51:57.401583 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:51:57.401583 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:51:57.401583 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:51:57.401583 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:51:57.401583 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:51:57.401583 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:51:57.401583 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:51:57.401583 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:51:57.401583 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:51:57.401583 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:51:57.401583 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:51:57.401583 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:51:57.401583 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:51:57.401583 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:51:57.401583 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:51:57.401583 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:51:57.402478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:51:57.402478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:51:57.402478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:51:57.402478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:51:57.402478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:51:57.402478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:51:57.402478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:51:57.402478 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:51:57.402478 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 635/1271] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:51:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:51:57.408105 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0996b3-b552-40da-91b8-c52746db7fda None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:57.466576 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:57.506349 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0996b3-b552-40da-91b8-c52746db7fda None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:57.540085 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609d74b-48f9-4750-9ace-f9a776aef0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:57.540713 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609d74b-48f9-4750-9ace-f9a776aef0f1 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:57.541015 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609d74b-48f9-4750-9ace-f9a776aef0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:57.546807 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609d74b-48f9-4750-9ace-f9a776aef0f1 None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:57.546807 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3609d74b-48f9-4750-9ace-f9a776aef0f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:51:57.546807 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3609d74b-48f9-4750-9ace-f9a776aef0f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:57.546807 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3609d74b-48f9-4750-9ace-f9a776aef0f1 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:57.557516 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 637/1272] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:51:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:51:57.563694 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a85d39-849f-44a4-90be-26bb86e9904e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:51:57.654292 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:51:57.663739 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a85d39-849f-44a4-90be-26bb86e9904e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:51:57.699792 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb3c8ec-a26e-4883-bfd1-0c28a887aa9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:51:57.700750 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3c8ec-a26e-4883-bfd1-0c28a887aa9e None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:57.701043 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3c8ec-a26e-4883-bfd1-0c28a887aa9e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:57.701329 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3c8ec-a26e-4883-bfd1-0c28a887aa9e None admin] PATH_INFO: `/v3/projects/5100a8d316d243fb8f82e9044e1b3b06/users/018802a8d9c041119ad708430f885a89/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:57.703774 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb3c8ec-a26e-4883-bfd1-0c28a887aa9e None admin] RBAC: Authorizing `identity:create_grant(project_id=5100a8d316d243fb8f82e9044e1b3b06, user_id=018802a8d9c041119ad708430f885a89, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:51:57.705426 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb3c8ec-a26e-4883-bfd1-0c28a887aa9e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:51:57.817017 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 636/1273] 10.222.0.27 () {58 vars in 1439 bytes} [Sun Sep 4 00:51:57 2022] PUT /identity/v3/projects/5100a8d316d243fb8f82e9044e1b3b06/users/018802a8d9c041119ad708430f885a89/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 00:51:59.740837 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabcb362-78a9-4d94-a91e-b7a0afc78102 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:59.741400 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabcb362-78a9-4d94-a91e-b7a0afc78102 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:59.741400 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabcb362-78a9-4d94-a91e-b7a0afc78102 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:59.742800 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 638/1274] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:51:59.748067 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9d7b6-c2b5-4e06-bfde-73574d764ca2 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:51:59.748512 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9d7b6-c2b5-4e06-bfde-73574d764ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:51:59.750639 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9d7b6-c2b5-4e06-bfde-73574d764ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:51:59.766645 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-b4e9d7b6-c2b5-4e06-bfde-73574d764ca2 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:51:59.768409 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b4e9d7b6-c2b5-4e06-bfde-73574d764ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:51:59.835752 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 637/1275] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:51:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:52:00.113566 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabcb362-78a9-4d94-a91e-b7a0afc78102 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:52:00.216826 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabcb362-78a9-4d94-a91e-b7a0afc78102 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:52:00.236602 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:52:00.253089 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b8e54-701a-4689-82a7-ccccab7b8a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:52:00.253709 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b8e54-701a-4689-82a7-ccccab7b8a6d None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:52:00.253990 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b8e54-701a-4689-82a7-ccccab7b8a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:52:00.254262 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b8e54-701a-4689-82a7-ccccab7b8a6d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:52:00.264855 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b8e54-701a-4689-82a7-ccccab7b8a6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 04 00:52:00.269642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-578b8e54-701a-4689-82a7-ccccab7b8a6d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:52:00.269642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:52:00.269642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:52:00.269642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:52:00.269642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:52:00.269642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:52:00.269642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:52:00.269642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:52:00.269642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:52:00.269642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:52:00.269642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:52:00.269642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:52:00.269642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 04 00:52:00.269642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 04 00:52:00.269642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 04 00:52:00.269642 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 04 00:52:00.273354 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 04 00:52:00.273354 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 04 00:52:00.273354 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 04 00:52:00.273354 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 04 00:52:00.273354 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:52:00.273354 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:52:00.273354 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:52:00.273354 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:52:00.273354 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:52:00.273354 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:52:00.273354 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:52:00.273354 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:52:00.273354 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:52:00.273354 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:52:00.273354 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:52:00.273354 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:52:00.274333 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:52:00.274333 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:52:00.274333 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:52:00.274333 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:52:00.274333 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 04 00:52:00.274333 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 04 00:52:00.274333 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 04 00:52:00.274333 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 04 00:52:00.274333 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 04 00:52:00.274333 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 04 00:52:00.274333 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 04 00:52:00.274333 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:52:00.274333 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 639/1276] 10.222.0.27 () {56 vars in 1096 bytes} [Sun Sep 4 00:52:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:52:00.276912 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e9d7b6-c2b5-4e06-bfde-73574d764ca2 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:52:00.358581 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:52:00.377415 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e9d7b6-c2b5-4e06-bfde-73574d764ca2 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:52:00.410217 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb0f393-b5ff-418a-98aa-a66b9c71b9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:52:00.410819 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0f393-b5ff-418a-98aa-a66b9c71b9b0 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:52:00.411102 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0f393-b5ff-418a-98aa-a66b9c71b9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:52:00.411376 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0f393-b5ff-418a-98aa-a66b9c71b9b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:52:00.412033 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb0f393-b5ff-418a-98aa-a66b9c71b9b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:52:00.412337 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb0f393-b5ff-418a-98aa-a66b9c71b9b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:52:00.413715 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb0f393-b5ff-418a-98aa-a66b9c71b9b0 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:52:00.421694 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 638/1277] 10.222.0.27 () {56 vars in 1099 bytes} [Sun Sep 4 00:52:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:52:00.428025 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b8e54-701a-4689-82a7-ccccab7b8a6d None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:52:00.529682 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b8e54-701a-4689-82a7-ccccab7b8a6d None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:52:00.562017 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b5f48-ea52-459c-b6d0-1e03ee312f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:52:00.562628 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b5f48-ea52-459c-b6d0-1e03ee312f31 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:52:00.562904 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b5f48-ea52-459c-b6d0-1e03ee312f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:52:00.563172 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b5f48-ea52-459c-b6d0-1e03ee312f31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:52:00.573883 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781b5f48-ea52-459c-b6d0-1e03ee312f31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:52:00.575318 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781b5f48-ea52-459c-b6d0-1e03ee312f31 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:52:00.588970 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-781b5f48-ea52-459c-b6d0-1e03ee312f31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:52:00.588970 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:52:00.588970 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:52:00.588970 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:52:00.588970 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:52:00.588970 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:52:00.588970 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:52:00.588970 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:52:00.588970 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:52:00.588970 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:52:00.588970 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:52:00.588970 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:52:00.588970 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:52:00.588970 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:52:00.588970 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:52:00.588970 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:52:00.590049 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:52:00.590049 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:52:00.590049 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:52:00.590049 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:52:00.590049 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:52:00.590049 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:52:00.590049 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:52:00.590049 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:52:00.590049 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:52:00.590049 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:52:00.590049 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:52:00.590049 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:52:00.590049 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:52:00.590049 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:52:00.590049 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:52:00.590049 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:52:00.590049 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:52:00.590974 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:52:00.590974 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:52:00.590974 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:52:00.590974 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 640/1278] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:52:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:52:00.596653 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb0f393-b5ff-418a-98aa-a66b9c71b9b0 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:52:00.696606 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb0f393-b5ff-418a-98aa-a66b9c71b9b0 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:52:00.729784 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3302f8cb-9636-48ec-875a-7a53abf79522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:52:00.730378 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3302f8cb-9636-48ec-875a-7a53abf79522 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:52:00.730658 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3302f8cb-9636-48ec-875a-7a53abf79522 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:52:00.730930 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3302f8cb-9636-48ec-875a-7a53abf79522 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:52:00.731560 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3302f8cb-9636-48ec-875a-7a53abf79522 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:52:00.731865 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3302f8cb-9636-48ec-875a-7a53abf79522 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:52:00.733234 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3302f8cb-9636-48ec-875a-7a53abf79522 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:52:00.745344 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 639/1279] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:52:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:52:00.751257 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b5f48-ea52-459c-b6d0-1e03ee312f31 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:52:00.781992 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:52:00.850378 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b5f48-ea52-459c-b6d0-1e03ee312f31 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:52:00.882974 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2369c67f-e4e5-4406-a3d0-ea168c4cb1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:52:00.883583 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2369c67f-e4e5-4406-a3d0-ea168c4cb1f4 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:52:00.883860 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2369c67f-e4e5-4406-a3d0-ea168c4cb1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:52:00.884131 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2369c67f-e4e5-4406-a3d0-ea168c4cb1f4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:52:00.890844 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2369c67f-e4e5-4406-a3d0-ea168c4cb1f4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:52:00.892351 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2369c67f-e4e5-4406-a3d0-ea168c4cb1f4 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:52:00.903469 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-2369c67f-e4e5-4406-a3d0-ea168c4cb1f4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 04 00:52:00.903469 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:52:00.903469 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:52:00.903469 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:52:00.903469 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:52:00.903469 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:52:00.903469 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:52:00.903469 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:52:00.903469 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:52:00.903469 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:52:00.903469 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:52:00.903469 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:52:00.903469 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 04 00:52:00.903469 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 04 00:52:00.903469 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 04 00:52:00.903469 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 04 00:52:00.912476 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:52:00.912476 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:52:00.912476 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 04 00:52:00.912476 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:52:00.912476 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 04 00:52:00.912476 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 04 00:52:00.912476 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:52:00.912476 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:52:00.912476 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:52:00.912476 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:52:00.912476 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:52:00.912476 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:52:00.912476 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:52:00.912476 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:52:00.912476 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:52:00.912476 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:52:00.912476 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:52:00.912476 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:52:00.913673 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:52:00.913673 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:52:00.913673 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 04 00:52:00.913673 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 04 00:52:00.913673 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 04 00:52:00.913673 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 04 00:52:00.913673 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 04 00:52:00.913673 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 04 00:52:00.913673 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 04 00:52:00.913673 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:52:00.913673 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 641/1280] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:52:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:52:00.914493 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3302f8cb-9636-48ec-875a-7a53abf79522 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:52:00.914998 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:52:01.009151 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3302f8cb-9636-48ec-875a-7a53abf79522 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:52:01.044676 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4ec43-dd4b-4ef5-bfc5-e3edcaf4fb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:52:01.045286 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4ec43-dd4b-4ef5-bfc5-e3edcaf4fb55 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:52:01.045568 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4ec43-dd4b-4ef5-bfc5-e3edcaf4fb55 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:52:01.045841 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4ec43-dd4b-4ef5-bfc5-e3edcaf4fb55 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:52:01.046560 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e4ec43-dd4b-4ef5-bfc5-e3edcaf4fb55 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:52:01.046866 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e4ec43-dd4b-4ef5-bfc5-e3edcaf4fb55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:52:01.048340 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e4ec43-dd4b-4ef5-bfc5-e3edcaf4fb55 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:52:01.066157 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 640/1281] 10.222.0.27 () {56 vars in 1143 bytes} [Sun Sep 4 00:52:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:52:01.072301 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2369c67f-e4e5-4406-a3d0-ea168c4cb1f4 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:52:01.163814 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:52:01.173394 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2369c67f-e4e5-4406-a3d0-ea168c4cb1f4 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:52:01.206502 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a269f72-9726-44ec-a2f2-452711728ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:52:01.207106 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a269f72-9726-44ec-a2f2-452711728ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:52:01.207385 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a269f72-9726-44ec-a2f2-452711728ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:52:01.207663 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a269f72-9726-44ec-a2f2-452711728ef4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:52:01.275224 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a269f72-9726-44ec-a2f2-452711728ef4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:52:01.276633 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a269f72-9726-44ec-a2f2-452711728ef4 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:52:01.369869 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:52:01.380082 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 642/1282] 10.222.0.27 () {58 vars in 1282 bytes} [Sun Sep 4 00:52:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 04 00:52:01.386006 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4ec43-dd4b-4ef5-bfc5-e3edcaf4fb55 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:52:01.447860 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:52:01.487362 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4ec43-dd4b-4ef5-bfc5-e3edcaf4fb55 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:52:01.521416 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7073e0-3f51-46d9-a1c9-b7bb3c66abb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:52:01.522038 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7073e0-3f51-46d9-a1c9-b7bb3c66abb4 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:52:01.522319 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7073e0-3f51-46d9-a1c9-b7bb3c66abb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:52:01.522593 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7073e0-3f51-46d9-a1c9-b7bb3c66abb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:52:01.533028 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7073e0-3f51-46d9-a1c9-b7bb3c66abb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:52:01.534655 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7073e0-3f51-46d9-a1c9-b7bb3c66abb4 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:52:01.548616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-db7073e0-3f51-46d9-a1c9-b7bb3c66abb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:52:01.548616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:52:01.548616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:52:01.548616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:52:01.548616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:52:01.548616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:52:01.548616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:52:01.548616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:52:01.548616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:52:01.548616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:52:01.548616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:52:01.548616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:52:01.548616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 04 00:52:01.548616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 04 00:52:01.548616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 04 00:52:01.548616 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 04 00:52:01.549776 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:52:01.549776 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:52:01.549776 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:52:01.549776 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:52:01.549776 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:52:01.549776 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:52:01.549776 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:52:01.549776 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:52:01.549776 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:52:01.549776 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:52:01.549776 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:52:01.549776 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:52:01.549776 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:52:01.549776 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:52:01.549776 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:52:01.549776 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:52:01.549776 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 04 00:52:01.550960 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 04 00:52:01.550960 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 04 00:52:01.550960 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:52:01.550960 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 641/1283] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 00:52:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:52:01.555498 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a269f72-9726-44ec-a2f2-452711728ef4 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:52:01.654665 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a269f72-9726-44ec-a2f2-452711728ef4 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:52:01.687869 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e15059-cff6-4e96-bb9c-e47a2c50d30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:52:01.688486 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e15059-cff6-4e96-bb9c-e47a2c50d30c None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:52:01.689491 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e15059-cff6-4e96-bb9c-e47a2c50d30c None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:52:01.689776 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e15059-cff6-4e96-bb9c-e47a2c50d30c None admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:52:01.690417 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e15059-cff6-4e96-bb9c-e47a2c50d30c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:52:01.690723 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e15059-cff6-4e96-bb9c-e47a2c50d30c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:52:01.692098 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e15059-cff6-4e96-bb9c-e47a2c50d30c None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:52:01.703127 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 643/1284] 10.222.0.27 () {56 vars in 1109 bytes} [Sun Sep 4 00:52:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:52:01.709629 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7073e0-3f51-46d9-a1c9-b7bb3c66abb4 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:52:01.807999 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7073e0-3f51-46d9-a1c9-b7bb3c66abb4 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:52:01.821722 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:52:01.842015 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf9af29-bcdc-4b9c-a81c-46abcf6092e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:52:01.842636 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf9af29-bcdc-4b9c-a81c-46abcf6092e7 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:52:01.842916 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf9af29-bcdc-4b9c-a81c-46abcf6092e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:52:01.843201 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf9af29-bcdc-4b9c-a81c-46abcf6092e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:52:01.850370 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf9af29-bcdc-4b9c-a81c-46abcf6092e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:52:01.851802 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf9af29-bcdc-4b9c-a81c-46abcf6092e7 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:52:01.863217 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-ecf9af29-bcdc-4b9c-a81c-46abcf6092e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:52:01.863217 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:52:01.863217 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:52:01.863217 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:52:01.863217 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:52:01.863217 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:52:01.863217 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:52:01.863217 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:52:01.863217 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:52:01.863217 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:52:01.863217 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:52:01.863217 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:52:01.863217 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 04 00:52:01.863217 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 04 00:52:01.863217 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 04 00:52:01.864211 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 04 00:52:01.864211 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:52:01.864211 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:52:01.864211 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:52:01.864211 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:52:01.864211 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:52:01.864211 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:52:01.864211 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:52:01.864211 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 04 00:52:01.864211 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:52:01.864211 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:52:01.864211 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:52:01.864211 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:52:01.864211 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:52:01.864211 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:52:01.864211 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:52:01.864211 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:52:01.868198 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 04 00:52:01.868198 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 04 00:52:01.868198 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 04 00:52:01.868198 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 04 00:52:01.868198 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 04 00:52:01.868198 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 04 00:52:01.868198 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 04 00:52:01.868198 dv-cinder-master-3-smb devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 04 00:52:01.868198 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 642/1285] 10.222.0.27 () {56 vars in 1111 bytes} [Sun Sep 4 00:52:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:52:01.870255 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e15059-cff6-4e96-bb9c-e47a2c50d30c None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:52:01.921187 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:52:01.970471 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e15059-cff6-4e96-bb9c-e47a2c50d30c None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:52:02.004066 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc9b01-c80d-4d4f-ad12-ccff514700f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:52:02.004697 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc9b01-c80d-4d4f-ad12-ccff514700f5 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:52:02.004982 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc9b01-c80d-4d4f-ad12-ccff514700f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:52:02.005259 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc9b01-c80d-4d4f-ad12-ccff514700f5 None admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:52:02.005948 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fc9b01-c80d-4d4f-ad12-ccff514700f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:52:02.006284 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fc9b01-c80d-4d4f-ad12-ccff514700f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:52:02.007731 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fc9b01-c80d-4d4f-ad12-ccff514700f5 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:52:02.020147 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 644/1286] 10.222.0.27 () {56 vars in 1148 bytes} [Sun Sep 4 00:52:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:52:02.026439 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf9af29-bcdc-4b9c-a81c-46abcf6092e7 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:52:02.125632 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf9af29-bcdc-4b9c-a81c-46abcf6092e7 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:52:02.160068 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f793fa-4c3e-4a44-bd2c-5850c50381a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:52:02.160677 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f793fa-4c3e-4a44-bd2c-5850c50381a6 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:52:02.160954 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f793fa-4c3e-4a44-bd2c-5850c50381a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:52:02.161225 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f793fa-4c3e-4a44-bd2c-5850c50381a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:52:02.161930 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f793fa-4c3e-4a44-bd2c-5850c50381a6 None admin] RBAC: Adding query filter params (role.id=c8a59d34ee2f4cefa6940cac778260c7, scope.project.id=5100a8d316d243fb8f82e9044e1b3b06, user.id=018802a8d9c041119ad708430f885a89) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:52:02.162231 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f793fa-4c3e-4a44-bd2c-5850c50381a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:52:02.163736 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f793fa-4c3e-4a44-bd2c-5850c50381a6 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:52:02.169589 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 643/1287] 10.222.0.27 () {56 vars in 1374 bytes} [Sun Sep 4 00:52:02 2022] GET /identity/v3/role_assignments?user.id=018802a8d9c041119ad708430f885a89&scope.project.id=5100a8d316d243fb8f82e9044e1b3b06&role.id=c8a59d34ee2f4cefa6940cac778260c7 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:52:04.102200 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b2a86-5cfa-4da6-bfa0-4475edc25e05 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:52:04.102733 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b2a86-5cfa-4da6-bfa0-4475edc25e05 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:52:04.102733 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b2a86-5cfa-4da6-bfa0-4475edc25e05 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:52:04.104080 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 645/1288] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:52:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:52:04.109124 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d0a96-213c-4d9a-9f83-21c0dd359f7c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:52:04.109537 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d0a96-213c-4d9a-9f83-21c0dd359f7c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:52:04.109701 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d0a96-213c-4d9a-9f83-21c0dd359f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:52:04.128853 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-219d0a96-213c-4d9a-9f83-21c0dd359f7c None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:52:04.130035 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-219d0a96-213c-4d9a-9f83-21c0dd359f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:52:04.159186 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:52:04.196140 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 644/1289] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:52:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:52:04.464411 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b2a86-5cfa-4da6-bfa0-4475edc25e05 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:52:04.565841 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b2a86-5cfa-4da6-bfa0-4475edc25e05 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:52:04.576311 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:52:04.599167 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef959322-0c9a-4435-9ba1-0ad4d61790d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:52:04.599785 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef959322-0c9a-4435-9ba1-0ad4d61790d5 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:52:04.600066 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef959322-0c9a-4435-9ba1-0ad4d61790d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:52:04.600343 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef959322-0c9a-4435-9ba1-0ad4d61790d5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:52:04.600951 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef959322-0c9a-4435-9ba1-0ad4d61790d5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:52:04.602318 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef959322-0c9a-4435-9ba1-0ad4d61790d5 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:52:04.611044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-ef959322-0c9a-4435-9ba1-0ad4d61790d5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 04 00:52:04.611044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:52:04.611044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 00:52:04.611044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 00:52:04.611044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 00:52:04.611044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 00:52:04.611044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 00:52:04.611044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:52:04.611044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 04 00:52:04.611044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:52:04.611044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 00:52:04.611044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:52:04.611044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:52:04.611044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:52:04.611044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:52:04.611044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:52:04.611044 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:52:04.619795 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:52:04.619795 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:52:04.619795 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:52:04.619795 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:52:04.619795 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:52:04.619795 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:52:04.619795 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 00:52:04.619795 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 00:52:04.619795 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 00:52:04.619795 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 00:52:04.619795 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:52:04.619795 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:52:04.619795 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:52:04.619795 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:52:04.619795 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:52:04.619795 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:52:04.620673 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:52:04.620673 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:52:04.620673 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:52:04.620673 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:52:04.620673 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:52:04.620673 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:52:04.620673 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:52:04.620673 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:52:04.620673 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:52:04.620673 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:52:04.620673 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 00:52:04.620673 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:52:04.620673 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 04 00:52:04.620673 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:52:04.620673 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 646/1290] 10.222.0.27 () {56 vars in 1117 bytes} [Sun Sep 4 00:52:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:52:04.621487 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d0a96-213c-4d9a-9f83-21c0dd359f7c None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:52:04.722324 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d0a96-213c-4d9a-9f83-21c0dd359f7c None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:52:04.759147 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cea8c80-1d3c-427e-a334-0ae3b9ee944c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:52:04.759760 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea8c80-1d3c-427e-a334-0ae3b9ee944c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:52:04.760042 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea8c80-1d3c-427e-a334-0ae3b9ee944c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:52:04.760312 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea8c80-1d3c-427e-a334-0ae3b9ee944c None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:52:04.760973 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cea8c80-1d3c-427e-a334-0ae3b9ee944c None admin] RBAC: Adding query filter params (name=placement) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:52:04.761271 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cea8c80-1d3c-427e-a334-0ae3b9ee944c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:52:04.762622 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cea8c80-1d3c-427e-a334-0ae3b9ee944c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:52:04.768441 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 645/1291] 10.222.0.27 () {56 vars in 1116 bytes} [Sun Sep 4 00:52:04 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:52:04.774836 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef959322-0c9a-4435-9ba1-0ad4d61790d5 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:52:04.883995 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef959322-0c9a-4435-9ba1-0ad4d61790d5 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:52:04.921269 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4526c0d-243e-482a-a6c2-58ce1320397a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:52:04.921883 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4526c0d-243e-482a-a6c2-58ce1320397a None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:52:04.922162 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4526c0d-243e-482a-a6c2-58ce1320397a None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:52:04.922432 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4526c0d-243e-482a-a6c2-58ce1320397a None admin] PATH_INFO: `/v3/services` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:52:04.923064 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4526c0d-243e-482a-a6c2-58ce1320397a None admin] RBAC: Adding query filter params (type=placement) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:52:04.923426 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4526c0d-243e-482a-a6c2-58ce1320397a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:52:04.926324 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4526c0d-243e-482a-a6c2-58ce1320397a None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:52:04.931735 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 647/1292] 10.222.0.27 () {56 vars in 1116 bytes} [Sun Sep 4 00:52:04 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:52:06.840978 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336b420-d4ff-437c-bd3c-4c8da551d71c None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:52:06.841531 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336b420-d4ff-437c-bd3c-4c8da551d71c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:52:06.841531 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336b420-d4ff-437c-bd3c-4c8da551d71c None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:52:06.842909 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 646/1293] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:52:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:52:06.848790 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7030fce-6a1a-481a-bf4f-8bfb308dd854 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:52:06.849195 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7030fce-6a1a-481a-bf4f-8bfb308dd854 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:52:06.849390 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7030fce-6a1a-481a-bf4f-8bfb308dd854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:52:06.868313 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-e7030fce-6a1a-481a-bf4f-8bfb308dd854 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:52:06.869545 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e7030fce-6a1a-481a-bf4f-8bfb308dd854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:52:06.898916 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:52:06.936378 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 648/1294] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:52:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 04 00:52:07.213913 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0336b420-d4ff-437c-bd3c-4c8da551d71c None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:52:07.245518 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:52:07.316605 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0336b420-d4ff-437c-bd3c-4c8da551d71c None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:52:07.349394 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1a583-abc6-4f3f-90d9-2c58498a2570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:52:07.349985 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1a583-abc6-4f3f-90d9-2c58498a2570 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:52:07.350262 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1a583-abc6-4f3f-90d9-2c58498a2570 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:52:07.350569 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1a583-abc6-4f3f-90d9-2c58498a2570 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:52:07.351195 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d1a583-abc6-4f3f-90d9-2c58498a2570 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:52:07.352554 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d1a583-abc6-4f3f-90d9-2c58498a2570 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:52:07.453186 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 647/1295] 10.222.0.27 () {60 vars in 1141 bytes} [Sun Sep 4 00:52:07 2022] POST /identity/v3/services => generated 246 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 00:52:09.358924 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15267476-825e-44ab-80d1-cf5c45b14379 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:52:09.359482 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15267476-825e-44ab-80d1-cf5c45b14379 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:52:09.359482 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15267476-825e-44ab-80d1-cf5c45b14379 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:52:09.360871 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 649/1296] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:52:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:52:09.366494 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc110a72-2024-4cb3-bbb4-8283342a5d95 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:52:09.366895 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc110a72-2024-4cb3-bbb4-8283342a5d95 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:52:09.367828 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc110a72-2024-4cb3-bbb4-8283342a5d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:52:09.385543 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-cc110a72-2024-4cb3-bbb4-8283342a5d95 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:52:09.386708 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-cc110a72-2024-4cb3-bbb4-8283342a5d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:52:09.458117 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 648/1297] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:52:09 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:52:09.729685 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15267476-825e-44ab-80d1-cf5c45b14379 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:52:09.829288 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15267476-825e-44ab-80d1-cf5c45b14379 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:52:09.864776 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44c74d-83d7-4f89-a4ba-0995a2340c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:52:09.865414 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44c74d-83d7-4f89-a4ba-0995a2340c08 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:52:09.865699 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44c74d-83d7-4f89-a4ba-0995a2340c08 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:52:09.865973 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44c74d-83d7-4f89-a4ba-0995a2340c08 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:52:09.866553 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a44c74d-83d7-4f89-a4ba-0995a2340c08 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:52:09.867947 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a44c74d-83d7-4f89-a4ba-0995a2340c08 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:52:09.876718 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-7a44c74d-83d7-4f89-a4ba-0995a2340c08 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 04 00:52:09.876718 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:52:09.876718 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 00:52:09.876718 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 00:52:09.876718 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 00:52:09.876718 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 00:52:09.876718 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 00:52:09.876718 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:52:09.876718 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 04 00:52:09.876718 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:52:09.876718 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 00:52:09.876718 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:52:09.876718 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:52:09.876718 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:52:09.876718 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:52:09.876718 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:52:09.876718 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:52:09.877901 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:52:09.877901 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:52:09.877901 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:52:09.877901 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:52:09.877901 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:52:09.877901 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:52:09.877901 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 00:52:09.877901 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 00:52:09.877901 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 00:52:09.877901 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 00:52:09.877901 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:52:09.877901 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:52:09.877901 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:52:09.877901 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:52:09.877901 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:52:09.877901 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:52:09.878762 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:52:09.878762 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:52:09.878762 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:52:09.878762 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:52:09.878762 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:52:09.878762 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:52:09.878762 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:52:09.878762 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:52:09.878762 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:52:09.878762 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:52:09.878762 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 00:52:09.878762 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:52:09.878762 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 04 00:52:09.878762 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:52:09.878762 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 650/1298] 10.222.0.27 () {56 vars in 1117 bytes} [Sun Sep 4 00:52:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:52:09.884033 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc110a72-2024-4cb3-bbb4-8283342a5d95 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:52:09.902370 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:52:09.982144 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc110a72-2024-4cb3-bbb4-8283342a5d95 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:52:10.015122 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14465259-d928-4900-b979-4e8c9e57a0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:52:10.015718 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14465259-d928-4900-b979-4e8c9e57a0f3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:52:10.016000 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14465259-d928-4900-b979-4e8c9e57a0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:52:10.016276 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14465259-d928-4900-b979-4e8c9e57a0f3 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:52:10.017015 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14465259-d928-4900-b979-4e8c9e57a0f3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:52:10.017385 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14465259-d928-4900-b979-4e8c9e57a0f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:52:10.019299 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14465259-d928-4900-b979-4e8c9e57a0f3 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:52:10.024346 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 649/1299] 10.222.0.27 () {56 vars in 1116 bytes} [Sun Sep 4 00:52:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:52:10.030439 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44c74d-83d7-4f89-a4ba-0995a2340c08 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:52:10.061613 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:52:10.128025 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44c74d-83d7-4f89-a4ba-0995a2340c08 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:52:10.162656 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce39cc-e556-4666-ba91-b75e94434f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:52:10.163319 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce39cc-e556-4666-ba91-b75e94434f6c None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:52:10.163603 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce39cc-e556-4666-ba91-b75e94434f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:52:10.163881 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce39cc-e556-4666-ba91-b75e94434f6c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:52:10.164607 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce39cc-e556-4666-ba91-b75e94434f6c None admin] RBAC: Adding query filter params (interface=public, service_id=4de34a0fd7604b3ba21d061b0592afff, region_id=RegionOne) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:52:10.164918 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce39cc-e556-4666-ba91-b75e94434f6c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:52:10.166290 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce39cc-e556-4666-ba91-b75e94434f6c None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:52:10.171291 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 651/1300] 10.222.0.27 () {56 vars in 1251 bytes} [Sun Sep 4 00:52:10 2022] GET /identity/v3/endpoints?service_id=4de34a0fd7604b3ba21d061b0592afff&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:52:10.177041 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14465259-d928-4900-b979-4e8c9e57a0f3 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:52:10.261765 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:52:10.279510 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14465259-d928-4900-b979-4e8c9e57a0f3 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:52:10.312297 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1cec6e-42b8-44af-8db3-4169e1cd8eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:52:10.312926 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1cec6e-42b8-44af-8db3-4169e1cd8eec None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:52:10.313206 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1cec6e-42b8-44af-8db3-4169e1cd8eec None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:52:10.313478 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1cec6e-42b8-44af-8db3-4169e1cd8eec None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:52:10.314068 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1cec6e-42b8-44af-8db3-4169e1cd8eec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:52:10.316059 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1cec6e-42b8-44af-8db3-4169e1cd8eec None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:52:10.322646 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 650/1301] 10.222.0.27 () {56 vars in 1087 bytes} [Sun Sep 4 00:52:10 2022] GET /identity/v3/services => generated 2191 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 00:52:12.222504 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae4837-e473-46c5-8bd5-f809d99e5045 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:52:12.223068 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae4837-e473-46c5-8bd5-f809d99e5045 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:52:12.223068 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae4837-e473-46c5-8bd5-f809d99e5045 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:52:12.224409 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 652/1302] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:52:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:52:12.229896 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340854e7-43ab-4fa0-a34d-45c22cb39647 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:52:12.230251 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340854e7-43ab-4fa0-a34d-45c22cb39647 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:52:12.230251 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340854e7-43ab-4fa0-a34d-45c22cb39647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:52:12.249024 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-340854e7-43ab-4fa0-a34d-45c22cb39647 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:52:12.250194 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-340854e7-43ab-4fa0-a34d-45c22cb39647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:52:12.316379 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 651/1303] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:52:12 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 04 00:52:12.581777 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cae4837-e473-46c5-8bd5-f809d99e5045 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:52:12.677094 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:52:12.683679 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cae4837-e473-46c5-8bd5-f809d99e5045 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:52:12.716432 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f63699-263d-428d-8d9d-a6e07daafe86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:52:12.717085 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f63699-263d-428d-8d9d-a6e07daafe86 None admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:52:12.717366 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f63699-263d-428d-8d9d-a6e07daafe86 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:52:12.717635 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f63699-263d-428d-8d9d-a6e07daafe86 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:52:12.718206 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f63699-263d-428d-8d9d-a6e07daafe86 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:52:12.719605 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f63699-263d-428d-8d9d-a6e07daafe86 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:52:12.728998 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application [None req-37f63699-263d-428d-8d9d-a6e07daafe86 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 04 00:52:12.728998 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:52:12.728998 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 04 00:52:12.728998 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 04 00:52:12.728998 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 04 00:52:12.728998 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 04 00:52:12.728998 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 04 00:52:12.728998 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:52:12.728998 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 04 00:52:12.728998 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:52:12.728998 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 04 00:52:12.728998 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:52:12.728998 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 04 00:52:12.728998 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 04 00:52:12.728998 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 04 00:52:12.728998 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 04 00:52:12.728998 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 04 00:52:12.730058 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 04 00:52:12.730058 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 04 00:52:12.730058 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 04 00:52:12.730058 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 04 00:52:12.730058 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 04 00:52:12.730058 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 04 00:52:12.730058 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 04 00:52:12.730058 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 04 00:52:12.730058 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 04 00:52:12.730058 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 04 00:52:12.730058 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 04 00:52:12.730058 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 04 00:52:12.730058 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 04 00:52:12.730058 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 04 00:52:12.730058 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 04 00:52:12.730058 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.get_or_create( Sep 04 00:52:12.731008 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 04 00:52:12.731008 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application with Lock( Sep 04 00:52:12.731008 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 04 00:52:12.731008 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self._enter() Sep 04 00:52:12.731008 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 04 00:52:12.731008 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 04 00:52:12.731008 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 04 00:52:12.731008 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application return self.creator() Sep 04 00:52:12.731008 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 04 00:52:12.731008 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application created_value = creator( Sep 04 00:52:12.731008 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 04 00:52:12.731008 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 04 00:52:12.731008 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 04 00:52:12.731008 dv-cinder-master-3-smb devstack@keystone.service[81630]: ERROR keystone.server.flask.application Sep 04 00:52:12.731008 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 653/1304] 10.222.0.27 () {56 vars in 1117 bytes} [Sun Sep 4 00:52:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 04 00:52:12.737413 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340854e7-43ab-4fa0-a34d-45c22cb39647 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:52:12.813483 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:52:12.836250 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340854e7-43ab-4fa0-a34d-45c22cb39647 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:52:12.872032 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6808128-d1ba-43d0-adcc-0502a9bb9d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:52:12.872658 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6808128-d1ba-43d0-adcc-0502a9bb9d31 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:52:12.872936 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6808128-d1ba-43d0-adcc-0502a9bb9d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:52:12.873208 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6808128-d1ba-43d0-adcc-0502a9bb9d31 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:52:12.873944 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6808128-d1ba-43d0-adcc-0502a9bb9d31 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 00:52:12.874283 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6808128-d1ba-43d0-adcc-0502a9bb9d31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:52:12.875642 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6808128-d1ba-43d0-adcc-0502a9bb9d31 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:52:12.880730 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 652/1305] 10.222.0.27 () {56 vars in 1116 bytes} [Sun Sep 4 00:52:12 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 00:52:12.886640 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f63699-263d-428d-8d9d-a6e07daafe86 None admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:52:12.987382 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f63699-263d-428d-8d9d-a6e07daafe86 None admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:52:13.020120 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa62ed-0bf0-4639-b078-1a3c0ff99de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:52:13.020742 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa62ed-0bf0-4639-b078-1a3c0ff99de0 None admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:52:13.021021 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa62ed-0bf0-4639-b078-1a3c0ff99de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:52:13.021292 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa62ed-0bf0-4639-b078-1a3c0ff99de0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:52:13.021949 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa62ed-0bf0-4639-b078-1a3c0ff99de0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:52:13.023318 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa62ed-0bf0-4639-b078-1a3c0ff99de0 None admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:52:13.110065 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 654/1306] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 00:52:12 2022] POST /identity/v3/endpoints => generated 331 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 00:52:22.877666 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=30 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:52:23.105520 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:58:08.698893 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fcd1e-9dda-4655-b478-f02d80327dfc None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:08.699937 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fcd1e-9dda-4655-b478-f02d80327dfc None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:08.700774 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fcd1e-9dda-4655-b478-f02d80327dfc None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:08.709772 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb3143-e76f-41d0-9500-b5954ccec82a None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:08.710711 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb3143-e76f-41d0-9500-b5954ccec82a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:08.711195 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb3143-e76f-41d0-9500-b5954ccec82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:08.712535 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 653/1307] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:58:08 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:58:08.844734 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-80cb3143-e76f-41d0-9500-b5954ccec82a None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:58:08.845983 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-80cb3143-e76f-41d0-9500-b5954ccec82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:08.962710 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 655/1308] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:58:09.269439 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457a49a-e76e-49ce-8a90-cbb14148a1c7 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:09.269727 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457a49a-e76e-49ce-8a90-cbb14148a1c7 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:09.270204 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457a49a-e76e-49ce-8a90-cbb14148a1c7 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:09.271536 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 654/1309] 10.222.0.27 () {54 vars in 965 bytes} [Sun Sep 4 00:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:58:09.277525 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d780414-9b95-431a-b33b-471e48797c3a None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:09.277813 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d780414-9b95-431a-b33b-471e48797c3a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:09.278086 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d780414-9b95-431a-b33b-471e48797c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:09.358337 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-3d780414-9b95-431a-b33b-471e48797c3a None None] MFA Rules not processed for user `861c278f8bf34b3bb407b2e051119ee9`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:58:09.359634 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3d780414-9b95-431a-b33b-471e48797c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:09.381213 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:58:09.438971 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 656/1310] 10.222.0.27 () {58 vars in 1062 bytes} [Sun Sep 4 00:58:09 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:58:09.447907 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f457a49a-e76e-49ce-8a90-cbb14148a1c7 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:58:09.449752 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f457a49a-e76e-49ce-8a90-cbb14148a1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:09.585113 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f457a49a-e76e-49ce-8a90-cbb14148a1c7 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:58:09.675552 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74484be-ff14-477f-9e1c-dd2af5f8e39f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:58:09.676161 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74484be-ff14-477f-9e1c-dd2af5f8e39f service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:09.676437 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74484be-ff14-477f-9e1c-dd2af5f8e39f service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:09.676736 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74484be-ff14-477f-9e1c-dd2af5f8e39f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:09.678378 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f74484be-ff14-477f-9e1c-dd2af5f8e39f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:09.736872 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:58:09.794376 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74484be-ff14-477f-9e1c-dd2af5f8e39f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:58:09.796176 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74484be-ff14-477f-9e1c-dd2af5f8e39f service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:58:09.816016 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 655/1311] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 00:58:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 00:58:12.061883 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9aa86d-de36-463b-bc63-8cab1f25bb75 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:12.062422 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9aa86d-de36-463b-bc63-8cab1f25bb75 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:12.062663 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9aa86d-de36-463b-bc63-8cab1f25bb75 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:12.064085 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 657/1312] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:58:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:58:12.069922 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ba5ef-0a06-4ab5-a656-8fd9488c9dc6 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:12.070444 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ba5ef-0a06-4ab5-a656-8fd9488c9dc6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:12.070561 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ba5ef-0a06-4ab5-a656-8fd9488c9dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:12.161752 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f06ba5ef-0a06-4ab5-a656-8fd9488c9dc6 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:58:12.163177 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f06ba5ef-0a06-4ab5-a656-8fd9488c9dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:12.268852 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 656/1313] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:58:12.572768 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683fd18e-7705-4386-9c5a-e2b2a0fbb61c None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:12.573056 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683fd18e-7705-4386-9c5a-e2b2a0fbb61c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:12.573336 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683fd18e-7705-4386-9c5a-e2b2a0fbb61c None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:12.574669 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 658/1314] 10.222.0.27 () {54 vars in 965 bytes} [Sun Sep 4 00:58:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:58:12.580962 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d446ac-abc8-49ef-9fed-1b7cf3f208fc None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:12.581250 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d446ac-abc8-49ef-9fed-1b7cf3f208fc None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:12.581527 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d446ac-abc8-49ef-9fed-1b7cf3f208fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:12.594454 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:58:12.658495 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-42d446ac-abc8-49ef-9fed-1b7cf3f208fc None None] MFA Rules not processed for user `861c278f8bf34b3bb407b2e051119ee9`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:58:12.659793 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-42d446ac-abc8-49ef-9fed-1b7cf3f208fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:12.733406 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 657/1315] 10.222.0.27 () {58 vars in 1062 bytes} [Sun Sep 4 00:58:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:58:12.742396 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683fd18e-7705-4386-9c5a-e2b2a0fbb61c None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:58:12.743910 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-683fd18e-7705-4386-9c5a-e2b2a0fbb61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:12.846978 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683fd18e-7705-4386-9c5a-e2b2a0fbb61c None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:58:12.916794 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:58:12.919177 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ee371-cee6-4384-af8a-9f4e3fdba468 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:58:12.919779 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ee371-cee6-4384-af8a-9f4e3fdba468 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:12.920062 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ee371-cee6-4384-af8a-9f4e3fdba468 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:12.920336 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ee371-cee6-4384-af8a-9f4e3fdba468 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:12.922009 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-471ee371-cee6-4384-af8a-9f4e3fdba468 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:13.030139 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471ee371-cee6-4384-af8a-9f4e3fdba468 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:58:13.031795 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471ee371-cee6-4384-af8a-9f4e3fdba468 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:58:13.050050 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 659/1316] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 00:58:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 00:58:15.659520 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8735d7-472b-4774-9528-992b0621ef3b None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:15.660048 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8735d7-472b-4774-9528-992b0621ef3b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:15.660159 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8735d7-472b-4774-9528-992b0621ef3b None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:15.661463 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 658/1317] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:58:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:58:15.666730 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50dd5a1-94ed-4b7d-a513-3edd508552c8 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:15.667105 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50dd5a1-94ed-4b7d-a513-3edd508552c8 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:15.667291 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50dd5a1-94ed-4b7d-a513-3edd508552c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:15.733222 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-f50dd5a1-94ed-4b7d-a513-3edd508552c8 None None] MFA Rules not processed for user `43855050fbcd47d3b5ded0e8ac79e20d`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:58:15.735342 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f50dd5a1-94ed-4b7d-a513-3edd508552c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:15.786561 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:58:15.831255 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 660/1318] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:58:15 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:58:16.116625 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8735d7-472b-4774-9528-992b0621ef3b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:58:16.118173 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8d8735d7-472b-4774-9528-992b0621ef3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:16.219952 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8735d7-472b-4774-9528-992b0621ef3b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:58:16.222824 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:58:16.295319 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e41348-1091-464c-80fa-5a6953db5666 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:58:16.295953 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e41348-1091-464c-80fa-5a6953db5666 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:16.296233 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e41348-1091-464c-80fa-5a6953db5666 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:16.296566 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e41348-1091-464c-80fa-5a6953db5666 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:16.298325 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d5e41348-1091-464c-80fa-5a6953db5666 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:16.402534 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e41348-1091-464c-80fa-5a6953db5666 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:58:16.404144 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e41348-1091-464c-80fa-5a6953db5666 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:58:16.422398 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 659/1319] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 00:58:16 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 00:58:19.047310 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeaa81a-db11-4aa1-b0a9-7428b9e38848 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:19.047855 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeaa81a-db11-4aa1-b0a9-7428b9e38848 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:19.047855 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeaa81a-db11-4aa1-b0a9-7428b9e38848 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:19.049246 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 661/1320] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:58:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:58:19.054428 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3265640-fd23-47f9-8230-08e44bd56f4d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:19.054824 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3265640-fd23-47f9-8230-08e44bd56f4d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:19.054981 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3265640-fd23-47f9-8230-08e44bd56f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:19.095667 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:58:19.120618 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-d3265640-fd23-47f9-8230-08e44bd56f4d None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:58:19.121817 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d3265640-fd23-47f9-8230-08e44bd56f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:19.217976 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 660/1321] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:58:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:58:19.503112 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeaa81a-db11-4aa1-b0a9-7428b9e38848 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:58:19.504683 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-4eeaa81a-db11-4aa1-b0a9-7428b9e38848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:19.606543 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeaa81a-db11-4aa1-b0a9-7428b9e38848 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:58:19.680292 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce50037-14b3-4c47-a5ac-3ce6df700151 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:58:19.680909 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce50037-14b3-4c47-a5ac-3ce6df700151 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:19.681189 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce50037-14b3-4c47-a5ac-3ce6df700151 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:19.681459 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce50037-14b3-4c47-a5ac-3ce6df700151 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:19.683089 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-fce50037-14b3-4c47-a5ac-3ce6df700151 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:19.690808 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:58:19.791615 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce50037-14b3-4c47-a5ac-3ce6df700151 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:58:19.793017 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce50037-14b3-4c47-a5ac-3ce6df700151 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:58:19.811400 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 662/1322] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 00:58:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 00:58:22.663867 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775dc19-8a00-458e-826b-85672fc6339e None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:22.664602 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775dc19-8a00-458e-826b-85672fc6339e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:22.664602 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775dc19-8a00-458e-826b-85672fc6339e None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:22.665807 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 661/1323] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:58:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:58:22.671012 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319fe6b-7ba6-4184-af71-7b7f2e859deb None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:22.671482 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319fe6b-7ba6-4184-af71-7b7f2e859deb None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:22.671887 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319fe6b-7ba6-4184-af71-7b7f2e859deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:22.739213 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-a319fe6b-7ba6-4184-af71-7b7f2e859deb None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:58:22.740475 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a319fe6b-7ba6-4184-af71-7b7f2e859deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:22.799259 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:58:22.839694 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 663/1324] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:58:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:58:23.127211 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a775dc19-8a00-458e-826b-85672fc6339e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:58:23.128778 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a775dc19-8a00-458e-826b-85672fc6339e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:23.232250 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a775dc19-8a00-458e-826b-85672fc6339e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:58:23.245149 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:58:23.307542 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6285398e-8a2f-46c9-af13-3fab0054520c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:58:23.308143 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6285398e-8a2f-46c9-af13-3fab0054520c service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:23.308418 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6285398e-8a2f-46c9-af13-3fab0054520c service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:23.308720 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6285398e-8a2f-46c9-af13-3fab0054520c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:23.310282 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6285398e-8a2f-46c9-af13-3fab0054520c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:23.420693 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6285398e-8a2f-46c9-af13-3fab0054520c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:58:23.422807 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6285398e-8a2f-46c9-af13-3fab0054520c service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:58:23.441507 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 662/1325] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 00:58:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 00:58:25.445369 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c0c26-2aa2-4db4-9a1b-9041d51285c3 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:25.445900 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c0c26-2aa2-4db4-9a1b-9041d51285c3 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:25.445900 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c0c26-2aa2-4db4-9a1b-9041d51285c3 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:25.447163 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 664/1326] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:58:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:58:25.452587 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86ec20-51f5-4d4d-8b1c-5378fd25cd7d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:25.453007 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86ec20-51f5-4d4d-8b1c-5378fd25cd7d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:25.453289 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86ec20-51f5-4d4d-8b1c-5378fd25cd7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:25.502649 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:58:25.520973 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-0e86ec20-51f5-4d4d-8b1c-5378fd25cd7d None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:58:25.522190 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0e86ec20-51f5-4d4d-8b1c-5378fd25cd7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:25.630724 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 663/1327] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:58:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:58:25.913051 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c0c26-2aa2-4db4-9a1b-9041d51285c3 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:58:25.914639 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d84c0c26-2aa2-4db4-9a1b-9041d51285c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:26.029807 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c0c26-2aa2-4db4-9a1b-9041d51285c3 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:58:26.105786 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c75a0-31a8-4e68-96a4-4724594e6ccf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:58:26.106406 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c75a0-31a8-4e68-96a4-4724594e6ccf service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:26.106688 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c75a0-31a8-4e68-96a4-4724594e6ccf service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:26.106965 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c75a0-31a8-4e68-96a4-4724594e6ccf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:26.108722 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5f9c75a0-31a8-4e68-96a4-4724594e6ccf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:26.126896 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:58:26.219338 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9c75a0-31a8-4e68-96a4-4724594e6ccf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:58:26.220959 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9c75a0-31a8-4e68-96a4-4724594e6ccf service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:58:26.241911 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 665/1328] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 00:58:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 00:58:28.203859 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b594a-8cb4-45be-96dc-96373396f2b2 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:28.204531 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b594a-8cb4-45be-96dc-96373396f2b2 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:28.204531 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b594a-8cb4-45be-96dc-96373396f2b2 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:28.205858 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 664/1329] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:58:28.211499 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29ba23-ddd0-4704-9328-d253bab8a5e4 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:28.212058 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29ba23-ddd0-4704-9328-d253bab8a5e4 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:28.212058 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29ba23-ddd0-4704-9328-d253bab8a5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:28.282481 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-7c29ba23-ddd0-4704-9328-d253bab8a5e4 None None] MFA Rules not processed for user `43855050fbcd47d3b5ded0e8ac79e20d`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:58:28.283686 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-7c29ba23-ddd0-4704-9328-d253bab8a5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:28.346515 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:58:28.378170 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 666/1330] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:58:28 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:58:28.663594 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77b594a-8cb4-45be-96dc-96373396f2b2 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:58:28.665165 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e77b594a-8cb4-45be-96dc-96373396f2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:28.778214 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77b594a-8cb4-45be-96dc-96373396f2b2 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:58:28.799562 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:58:28.855954 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa22ae-33d9-46a0-9694-eb08042dd669 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:58:28.856531 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa22ae-33d9-46a0-9694-eb08042dd669 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:28.856812 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa22ae-33d9-46a0-9694-eb08042dd669 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:28.857080 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa22ae-33d9-46a0-9694-eb08042dd669 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:28.859166 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-97fa22ae-33d9-46a0-9694-eb08042dd669 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:28.969651 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa22ae-33d9-46a0-9694-eb08042dd669 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:58:28.971096 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa22ae-33d9-46a0-9694-eb08042dd669 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:58:28.989956 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 665/1331] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 00:58:28 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 00:58:31.258782 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73a1e0-8703-4499-863b-9df2d94fa725 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:31.259254 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73a1e0-8703-4499-863b-9df2d94fa725 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:31.259366 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73a1e0-8703-4499-863b-9df2d94fa725 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:31.260711 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 667/1332] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:58:31.266726 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8e8dc-ae83-4fcb-95e9-e052b523bf01 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:31.267175 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8e8dc-ae83-4fcb-95e9-e052b523bf01 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:31.267506 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8e8dc-ae83-4fcb-95e9-e052b523bf01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:31.331863 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:58:31.334666 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-ccc8e8dc-ae83-4fcb-95e9-e052b523bf01 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:58:31.335859 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ccc8e8dc-ae83-4fcb-95e9-e052b523bf01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:31.430375 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 666/1333] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:58:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:58:31.713631 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a73a1e0-8703-4499-863b-9df2d94fa725 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:58:31.715193 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8a73a1e0-8703-4499-863b-9df2d94fa725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:31.818265 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a73a1e0-8703-4499-863b-9df2d94fa725 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:58:31.892252 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6f7852-2848-42ee-98d4-19405cf3b586 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:58:31.892918 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f7852-2848-42ee-98d4-19405cf3b586 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:31.893199 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f7852-2848-42ee-98d4-19405cf3b586 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:31.893487 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f7852-2848-42ee-98d4-19405cf3b586 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:31.895071 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-fc6f7852-2848-42ee-98d4-19405cf3b586 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:31.922334 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:58:32.003684 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6f7852-2848-42ee-98d4-19405cf3b586 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:58:32.005160 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6f7852-2848-42ee-98d4-19405cf3b586 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:58:32.023518 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 668/1334] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 00:58:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 00:58:33.975784 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b198ec-8d92-4fe5-b27f-1db46932f521 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:33.976312 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b198ec-8d92-4fe5-b27f-1db46932f521 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:33.976430 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b198ec-8d92-4fe5-b27f-1db46932f521 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:33.977753 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 667/1335] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:58:33.982923 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9795c-477b-432c-ac17-6d3b605e97f8 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:33.983926 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9795c-477b-432c-ac17-6d3b605e97f8 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:33.984142 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9795c-477b-432c-ac17-6d3b605e97f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:34.048909 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-f9c9795c-477b-432c-ac17-6d3b605e97f8 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:58:34.050106 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f9c9795c-477b-432c-ac17-6d3b605e97f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:34.124521 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:58:34.146711 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 669/1336] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:58:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:58:34.432113 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b198ec-8d92-4fe5-b27f-1db46932f521 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:58:34.433670 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-13b198ec-8d92-4fe5-b27f-1db46932f521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:34.538094 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b198ec-8d92-4fe5-b27f-1db46932f521 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:58:34.566737 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:58:34.613168 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577bb4c-233a-480b-8e36-3cbf4ebd6cc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:58:34.613771 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577bb4c-233a-480b-8e36-3cbf4ebd6cc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:34.614054 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577bb4c-233a-480b-8e36-3cbf4ebd6cc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:34.614329 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577bb4c-233a-480b-8e36-3cbf4ebd6cc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:34.615915 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9577bb4c-233a-480b-8e36-3cbf4ebd6cc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:34.726443 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9577bb4c-233a-480b-8e36-3cbf4ebd6cc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:58:34.727915 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9577bb4c-233a-480b-8e36-3cbf4ebd6cc8 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:58:34.862280 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 668/1337] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 00:58:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 00:58:37.287942 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed10e28c-1690-49e8-a0af-62f7d44b3b10 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:37.288506 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed10e28c-1690-49e8-a0af-62f7d44b3b10 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:37.288506 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed10e28c-1690-49e8-a0af-62f7d44b3b10 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:37.289900 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 670/1338] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:58:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:58:37.295190 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389b0be-68b8-41d2-9baa-b59b5ccd28b6 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:37.295581 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389b0be-68b8-41d2-9baa-b59b5ccd28b6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:37.295729 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389b0be-68b8-41d2-9baa-b59b5ccd28b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:37.362770 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f389b0be-68b8-41d2-9baa-b59b5ccd28b6 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:58:37.363962 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f389b0be-68b8-41d2-9baa-b59b5ccd28b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:37.372111 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:58:37.461002 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 669/1339] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:58:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:58:37.749674 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed10e28c-1690-49e8-a0af-62f7d44b3b10 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:58:37.751423 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ed10e28c-1690-49e8-a0af-62f7d44b3b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:37.854840 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed10e28c-1690-49e8-a0af-62f7d44b3b10 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:58:37.930420 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439b8172-99aa-4fa4-9479-49304209f1aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:58:37.931023 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b8172-99aa-4fa4-9479-49304209f1aa service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:37.931301 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b8172-99aa-4fa4-9479-49304209f1aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:37.931572 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b8172-99aa-4fa4-9479-49304209f1aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:37.933217 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-439b8172-99aa-4fa4-9479-49304209f1aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:37.969113 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:58:38.044703 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439b8172-99aa-4fa4-9479-49304209f1aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:58:38.046237 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439b8172-99aa-4fa4-9479-49304209f1aa service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:58:38.064041 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 671/1340] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 00:58:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 00:58:42.118684 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33280cf8-1e89-445f-a9c0-2db8e4739be5 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:42.119166 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33280cf8-1e89-445f-a9c0-2db8e4739be5 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:42.119273 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33280cf8-1e89-445f-a9c0-2db8e4739be5 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:42.120910 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 670/1341] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:58:42.126247 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bf2a8-523f-4ea4-b4b9-8641b3844b2b None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:42.126686 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bf2a8-523f-4ea4-b4b9-8641b3844b2b None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:42.126832 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bf2a8-523f-4ea4-b4b9-8641b3844b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:42.194261 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-876bf2a8-523f-4ea4-b4b9-8641b3844b2b None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:58:42.195489 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-876bf2a8-523f-4ea4-b4b9-8641b3844b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:42.287199 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:58:42.299888 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 672/1342] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:58:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:58:42.594216 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33280cf8-1e89-445f-a9c0-2db8e4739be5 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:58:42.595764 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-33280cf8-1e89-445f-a9c0-2db8e4739be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:42.698512 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33280cf8-1e89-445f-a9c0-2db8e4739be5 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:58:42.735818 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:58:42.773075 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e45a6b-66a9-4379-9715-7fdcafd984a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:58:42.773695 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e45a6b-66a9-4379-9715-7fdcafd984a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:42.773972 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e45a6b-66a9-4379-9715-7fdcafd984a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:42.774246 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e45a6b-66a9-4379-9715-7fdcafd984a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:42.776083 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-67e45a6b-66a9-4379-9715-7fdcafd984a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:42.886355 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e45a6b-66a9-4379-9715-7fdcafd984a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:58:42.887694 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e45a6b-66a9-4379-9715-7fdcafd984a6 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:58:42.907012 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 671/1343] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 00:58:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 00:58:45.863553 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37143e-c344-42a5-a934-9da6612fbd14 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:45.864088 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37143e-c344-42a5-a934-9da6612fbd14 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:45.864088 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37143e-c344-42a5-a934-9da6612fbd14 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:45.865365 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 673/1344] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:58:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:58:45.870463 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c3ef5-310d-444b-a164-d2bb4c0975b4 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:45.870912 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c3ef5-310d-444b-a164-d2bb4c0975b4 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:45.871149 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c3ef5-310d-444b-a164-d2bb4c0975b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:45.938660 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-437c3ef5-310d-444b-a164-d2bb4c0975b4 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:58:45.939850 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-437c3ef5-310d-444b-a164-d2bb4c0975b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:45.958437 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:58:46.039047 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 672/1345] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:58:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:58:46.327399 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37143e-c344-42a5-a934-9da6612fbd14 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:58:46.329073 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-4a37143e-c344-42a5-a934-9da6612fbd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:46.437645 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37143e-c344-42a5-a934-9da6612fbd14 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:58:46.516835 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f3383-94ba-42fa-8f9c-40a26a1cd79c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:58:46.517446 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f3383-94ba-42fa-8f9c-40a26a1cd79c service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:46.517726 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f3383-94ba-42fa-8f9c-40a26a1cd79c service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:46.517999 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f3383-94ba-42fa-8f9c-40a26a1cd79c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:46.519605 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d71f3383-94ba-42fa-8f9c-40a26a1cd79c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:46.563642 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:58:46.633495 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f3383-94ba-42fa-8f9c-40a26a1cd79c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:58:46.634976 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f3383-94ba-42fa-8f9c-40a26a1cd79c service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:58:46.654117 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 674/1346] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 00:58:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 00:58:50.774650 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b82c6-c56e-4b26-838e-060049187f99 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:50.775172 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b82c6-c56e-4b26-838e-060049187f99 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:50.775172 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b82c6-c56e-4b26-838e-060049187f99 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:50.776777 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 673/1347] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:58:50.782410 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc94627-8285-4fae-9c24-2fe9b1a3e25d None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:50.782924 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc94627-8285-4fae-9c24-2fe9b1a3e25d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:50.783060 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc94627-8285-4fae-9c24-2fe9b1a3e25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:50.852597 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-7bc94627-8285-4fae-9c24-2fe9b1a3e25d None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:58:50.853851 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-7bc94627-8285-4fae-9c24-2fe9b1a3e25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:50.954136 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:58:50.956875 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 675/1348] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:58:51.246901 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7b82c6-c56e-4b26-838e-060049187f99 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:58:51.249029 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6b7b82c6-c56e-4b26-838e-060049187f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:51.352437 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7b82c6-c56e-4b26-838e-060049187f99 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:58:51.399871 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:58:51.426040 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28bb4c4-f6c1-4d06-beb2-eb5ded74fb19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:58:51.426646 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28bb4c4-f6c1-4d06-beb2-eb5ded74fb19 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:51.426926 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28bb4c4-f6c1-4d06-beb2-eb5ded74fb19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:51.427201 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28bb4c4-f6c1-4d06-beb2-eb5ded74fb19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:51.428827 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d28bb4c4-f6c1-4d06-beb2-eb5ded74fb19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:51.535497 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28bb4c4-f6c1-4d06-beb2-eb5ded74fb19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:58:51.536959 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28bb4c4-f6c1-4d06-beb2-eb5ded74fb19 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:58:51.555401 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 674/1349] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 00:58:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 00:58:56.631978 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e25948-10bf-4343-9a83-3642242d8a58 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:56.632391 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e25948-10bf-4343-9a83-3642242d8a58 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:56.632549 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e25948-10bf-4343-9a83-3642242d8a58 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:56.634031 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 676/1350] 10.222.0.27 () {54 vars in 921 bytes} [Sun Sep 4 00:58:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:58:56.640053 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e646afe-05ff-4cf4-bd69-bf7478e7f879 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:58:56.640492 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e646afe-05ff-4cf4-bd69-bf7478e7f879 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:58:56.640693 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e646afe-05ff-4cf4-bd69-bf7478e7f879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:58:56.774598 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-0e646afe-05ff-4cf4-bd69-bf7478e7f879 None None] MFA Rules not processed for user `018802a8d9c041119ad708430f885a89`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:58:56.775999 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0e646afe-05ff-4cf4-bd69-bf7478e7f879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:58:56.805614 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:58:56.917645 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 675/1351] 10.222.0.27 () {58 vars in 1018 bytes} [Sun Sep 4 00:58:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:59:01.130025 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe749d-23f4-4f34-ba9e-97d3ee33fa2c None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:01.130609 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe749d-23f4-4f34-ba9e-97d3ee33fa2c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:01.130609 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe749d-23f4-4f34-ba9e-97d3ee33fa2c None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:01.132177 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 677/1352] 10.222.0.27 () {54 vars in 921 bytes} [Sun Sep 4 00:59:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:59:01.136992 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eeb07f-a752-4405-9ef8-25a6b722159b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:01.137401 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eeb07f-a752-4405-9ef8-25a6b722159b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:01.137607 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eeb07f-a752-4405-9ef8-25a6b722159b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:01.216197 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-e8eeb07f-a752-4405-9ef8-25a6b722159b None None] MFA Rules not processed for user `018802a8d9c041119ad708430f885a89`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:59:01.217464 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e8eeb07f-a752-4405-9ef8-25a6b722159b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:01.297094 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:01.337030 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 676/1353] 10.222.0.27 () {58 vars in 1018 bytes} [Sun Sep 4 00:59:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:59:02.652152 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8aa09b-68c4-4f7b-924b-c0d6be28a320 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:02.652732 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8aa09b-68c4-4f7b-924b-c0d6be28a320 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:02.652732 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8aa09b-68c4-4f7b-924b-c0d6be28a320 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:02.654221 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 678/1354] 10.222.0.27 () {54 vars in 921 bytes} [Sun Sep 4 00:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 00:59:02.660240 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff4b3b-bab7-4b51-8379-41bd9159e603 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:02.660686 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff4b3b-bab7-4b51-8379-41bd9159e603 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:02.660871 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff4b3b-bab7-4b51-8379-41bd9159e603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:02.740656 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-36ff4b3b-bab7-4b51-8379-41bd9159e603 None None] MFA Rules not processed for user `018802a8d9c041119ad708430f885a89`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:59:02.741870 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-36ff4b3b-bab7-4b51-8379-41bd9159e603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:02.849794 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 677/1355] 10.222.0.27 () {58 vars in 1018 bytes} [Sun Sep 4 00:59:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:59:06.244879 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17aa67-fd9f-40b6-99a7-95b4b69fbb99 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:06.245364 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17aa67-fd9f-40b6-99a7-95b4b69fbb99 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:06.245462 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17aa67-fd9f-40b6-99a7-95b4b69fbb99 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:06.248590 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 679/1356] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:59:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:59:06.252886 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8ad72-7533-4aaa-9a69-f3237f27b063 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:06.253311 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8ad72-7533-4aaa-9a69-f3237f27b063 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:06.253433 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8ad72-7533-4aaa-9a69-f3237f27b063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:06.297397 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:06.345871 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-95d8ad72-7533-4aaa-9a69-f3237f27b063 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:59:06.347110 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-95d8ad72-7533-4aaa-9a69-f3237f27b063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:06.470468 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 678/1357] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:59:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:59:06.772127 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65421f90-a8da-4a43-94b5-37d5b0ea5d86 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:06.772354 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65421f90-a8da-4a43-94b5-37d5b0ea5d86 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:06.772665 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65421f90-a8da-4a43-94b5-37d5b0ea5d86 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:06.773999 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 680/1358] 10.222.0.27 () {54 vars in 962 bytes} [Sun Sep 4 00:59:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:59:06.783492 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c28cfa-c627-438b-a005-ff32595a75f5 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:06.784024 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c28cfa-c627-438b-a005-ff32595a75f5 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:06.784391 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c28cfa-c627-438b-a005-ff32595a75f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:06.864875 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-a0c28cfa-c627-438b-a005-ff32595a75f5 None None] MFA Rules not processed for user `68bc00bda5194cd8a7928f8c4df4ccd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:59:06.866172 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a0c28cfa-c627-438b-a005-ff32595a75f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:06.928284 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:07.008878 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 679/1359] 10.222.0.27 () {58 vars in 1059 bytes} [Sun Sep 4 00:59:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:59:07.010365 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65421f90-a8da-4a43-94b5-37d5b0ea5d86 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:59:07.012291 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-65421f90-a8da-4a43-94b5-37d5b0ea5d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:07.180748 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65421f90-a8da-4a43-94b5-37d5b0ea5d86 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:59:07.264773 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:07.308778 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e76fdda-7475-4214-92d1-6de184cc03cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:59:07.309504 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76fdda-7475-4214-92d1-6de184cc03cb service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:07.309880 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76fdda-7475-4214-92d1-6de184cc03cb service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:07.310254 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76fdda-7475-4214-92d1-6de184cc03cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:07.312068 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5e76fdda-7475-4214-92d1-6de184cc03cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:07.408212 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e76fdda-7475-4214-92d1-6de184cc03cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:59:07.410198 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e76fdda-7475-4214-92d1-6de184cc03cb service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:59:07.431453 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 681/1360] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 00:59:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 00:59:09.588383 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36293db9-fa39-41e8-84f5-7fc89a0fb067 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:09.588986 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36293db9-fa39-41e8-84f5-7fc89a0fb067 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:09.588986 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36293db9-fa39-41e8-84f5-7fc89a0fb067 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:09.590482 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 680/1361] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:59:09.596214 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6af546-dd29-4689-aab5-0384b5598e83 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:09.597045 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6af546-dd29-4689-aab5-0384b5598e83 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:09.597323 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6af546-dd29-4689-aab5-0384b5598e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:09.666775 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-2c6af546-dd29-4689-aab5-0384b5598e83 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:59:09.667903 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2c6af546-dd29-4689-aab5-0384b5598e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:09.707766 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:09.781613 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 682/1362] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:59:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:59:10.074436 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36293db9-fa39-41e8-84f5-7fc89a0fb067 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:59:10.077334 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-36293db9-fa39-41e8-84f5-7fc89a0fb067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:10.236396 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36293db9-fa39-41e8-84f5-7fc89a0fb067 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:59:10.246637 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:10.356136 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a6831-a763-49d2-befe-e9af31df3338 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:59:10.356966 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a6831-a763-49d2-befe-e9af31df3338 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:10.357506 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a6831-a763-49d2-befe-e9af31df3338 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:10.358061 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a6831-a763-49d2-befe-e9af31df3338 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:10.359775 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e60a6831-a763-49d2-befe-e9af31df3338 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:10.468069 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60a6831-a763-49d2-befe-e9af31df3338 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:59:10.469916 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60a6831-a763-49d2-befe-e9af31df3338 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:59:10.490133 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 681/1363] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 00:59:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 00:59:12.864348 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7ba42-9e33-47e4-a65f-bea7ab2be71e None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:12.865290 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7ba42-9e33-47e4-a65f-bea7ab2be71e None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:12.865723 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7ba42-9e33-47e4-a65f-bea7ab2be71e None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:12.867236 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 683/1364] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:59:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:59:12.872451 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bdd748-4392-45ff-a976-a61c3346706c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:12.873756 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bdd748-4392-45ff-a976-a61c3346706c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:12.874190 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bdd748-4392-45ff-a976-a61c3346706c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:12.890942 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:12.946345 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-75bdd748-4392-45ff-a976-a61c3346706c None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:59:12.947709 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-75bdd748-4392-45ff-a976-a61c3346706c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:13.053048 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 682/1365] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:59:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:59:13.353433 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ccf006-2965-4478-9515-fd94ae058955 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:13.353947 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ccf006-2965-4478-9515-fd94ae058955 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:13.354378 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ccf006-2965-4478-9515-fd94ae058955 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:13.355845 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 684/1366] 10.222.0.27 () {54 vars in 962 bytes} [Sun Sep 4 00:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:59:13.362464 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41bc053-49f6-47ff-9e61-d302372a8f17 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:13.369557 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41bc053-49f6-47ff-9e61-d302372a8f17 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:13.369557 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41bc053-49f6-47ff-9e61-d302372a8f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:13.447629 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c41bc053-49f6-47ff-9e61-d302372a8f17 None None] MFA Rules not processed for user `68bc00bda5194cd8a7928f8c4df4ccd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:59:13.448906 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c41bc053-49f6-47ff-9e61-d302372a8f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:13.490561 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:13.587191 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 683/1367] 10.222.0.27 () {58 vars in 1059 bytes} [Sun Sep 4 00:59:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:59:13.596283 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ccf006-2965-4478-9515-fd94ae058955 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:59:13.597897 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-65ccf006-2965-4478-9515-fd94ae058955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:13.754748 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ccf006-2965-4478-9515-fd94ae058955 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:59:13.776421 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:13.880962 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ada39-5893-4c95-8a7b-29af734c545e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:59:13.881630 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ada39-5893-4c95-8a7b-29af734c545e service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:13.881908 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ada39-5893-4c95-8a7b-29af734c545e service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:13.882110 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ada39-5893-4c95-8a7b-29af734c545e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:13.883727 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-825ada39-5893-4c95-8a7b-29af734c545e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:13.977389 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ada39-5893-4c95-8a7b-29af734c545e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:59:13.979011 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ada39-5893-4c95-8a7b-29af734c545e service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:59:13.998368 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 685/1368] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 00:59:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 00:59:14.442450 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76558be8-192b-4db1-b61c-19548036135e None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:14.442853 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76558be8-192b-4db1-b61c-19548036135e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:14.442967 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76558be8-192b-4db1-b61c-19548036135e None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:14.444425 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 684/1369] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:59:14.449498 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0beaa-541f-4321-83da-4673c55b45aa None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:14.449789 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0beaa-541f-4321-83da-4673c55b45aa None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:14.450069 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0beaa-541f-4321-83da-4673c55b45aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:14.483028 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:14.523223 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-c9d0beaa-541f-4321-83da-4673c55b45aa None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:59:14.524573 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c9d0beaa-541f-4321-83da-4673c55b45aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:14.627148 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 686/1370] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:59:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:59:15.236191 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4c456-e9a9-4aa2-a41d-2ff37d354880 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:15.236565 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4c456-e9a9-4aa2-a41d-2ff37d354880 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:15.236881 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4c456-e9a9-4aa2-a41d-2ff37d354880 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:15.238389 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 685/1371] 10.222.0.27 () {54 vars in 964 bytes} [Sun Sep 4 00:59:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:59:15.243516 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2c2d9-fb0f-4f02-bce5-2db8615b126c None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:15.244012 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2c2d9-fb0f-4f02-bce5-2db8615b126c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:15.244099 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2c2d9-fb0f-4f02-bce5-2db8615b126c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:15.330316 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-94b2c2d9-fb0f-4f02-bce5-2db8615b126c None None] MFA Rules not processed for user `6297dbc521e94330ae9294b942144181`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:59:15.332026 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-94b2c2d9-fb0f-4f02-bce5-2db8615b126c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:15.382720 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:15.415539 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 687/1372] 10.222.0.27 () {58 vars in 1061 bytes} [Sun Sep 4 00:59:15 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:59:15.423666 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a4c456-e9a9-4aa2-a41d-2ff37d354880 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:59:15.425206 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-79a4c456-e9a9-4aa2-a41d-2ff37d354880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:15.536977 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a4c456-e9a9-4aa2-a41d-2ff37d354880 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:59:15.584931 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:15.620233 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07634cca-661a-454f-8681-cfd7f9ced01d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:59:15.621000 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07634cca-661a-454f-8681-cfd7f9ced01d service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:15.621362 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07634cca-661a-454f-8681-cfd7f9ced01d service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:15.621637 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07634cca-661a-454f-8681-cfd7f9ced01d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:15.623923 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-07634cca-661a-454f-8681-cfd7f9ced01d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:15.761452 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07634cca-661a-454f-8681-cfd7f9ced01d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:59:15.764036 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07634cca-661a-454f-8681-cfd7f9ced01d service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:59:15.793509 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 686/1373] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 00:59:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 00:59:16.114107 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d068d18-b713-4384-8ea6-2960dcab5a56 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:16.114396 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d068d18-b713-4384-8ea6-2960dcab5a56 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:16.114675 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d068d18-b713-4384-8ea6-2960dcab5a56 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:16.116033 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 688/1374] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:59:16.121268 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a6cf0-092a-45b0-b671-ecd66ef35926 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:16.121555 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a6cf0-092a-45b0-b671-ecd66ef35926 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:16.121827 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a6cf0-092a-45b0-b671-ecd66ef35926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:16.189962 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-ba9a6cf0-092a-45b0-b671-ecd66ef35926 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:59:16.191082 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ba9a6cf0-092a-45b0-b671-ecd66ef35926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:16.212568 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:16.293557 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 687/1375] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:59:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:59:16.583411 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d068d18-b713-4384-8ea6-2960dcab5a56 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:59:16.584970 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3d068d18-b713-4384-8ea6-2960dcab5a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:16.728798 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d068d18-b713-4384-8ea6-2960dcab5a56 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:59:16.779744 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:16.842783 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1ac6d-066d-4e97-ade7-6946b43398b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:59:16.843386 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1ac6d-066d-4e97-ade7-6946b43398b8 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:16.843670 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1ac6d-066d-4e97-ade7-6946b43398b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:16.843940 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1ac6d-066d-4e97-ade7-6946b43398b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:16.845575 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-11c1ac6d-066d-4e97-ade7-6946b43398b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:16.934589 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1ac6d-066d-4e97-ade7-6946b43398b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:59:16.936230 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1ac6d-066d-4e97-ade7-6946b43398b8 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:59:16.955182 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 689/1376] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 00:59:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 00:59:19.022909 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31fd090-901c-4119-9109-ae20a68ea8b3 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:19.023403 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31fd090-901c-4119-9109-ae20a68ea8b3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:19.023403 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31fd090-901c-4119-9109-ae20a68ea8b3 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:19.025155 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 688/1377] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:59:19.030707 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e471a-9b67-482a-9235-8c162607a834 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:19.031131 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e471a-9b67-482a-9235-8c162607a834 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:19.031233 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e471a-9b67-482a-9235-8c162607a834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:19.097646 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-d06e471a-9b67-482a-9235-8c162607a834 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:59:19.098840 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d06e471a-9b67-482a-9235-8c162607a834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:19.105380 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:19.197324 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 690/1378] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:59:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:59:19.506241 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31fd090-901c-4119-9109-ae20a68ea8b3 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:59:19.508296 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e31fd090-901c-4119-9109-ae20a68ea8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:19.676999 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31fd090-901c-4119-9109-ae20a68ea8b3 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:59:19.680947 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:19.797493 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639e0b3-5a4a-4615-a52a-bbf9411577d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:59:19.798234 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639e0b3-5a4a-4615-a52a-bbf9411577d0 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:19.798513 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639e0b3-5a4a-4615-a52a-bbf9411577d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:19.798788 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639e0b3-5a4a-4615-a52a-bbf9411577d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:19.801130 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5639e0b3-5a4a-4615-a52a-bbf9411577d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:19.899389 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5639e0b3-5a4a-4615-a52a-bbf9411577d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:59:19.901292 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5639e0b3-5a4a-4615-a52a-bbf9411577d0 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:59:19.921383 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 689/1379] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 00:59:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 00:59:22.015262 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71cc62-0ead-4187-8c6c-4a7302fcf1c3 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:22.015809 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71cc62-0ead-4187-8c6c-4a7302fcf1c3 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:22.015809 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71cc62-0ead-4187-8c6c-4a7302fcf1c3 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:22.017315 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 691/1380] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:59:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:59:22.022385 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9976a-3a37-42bc-8202-54adb9fb9e91 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:22.022823 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9976a-3a37-42bc-8202-54adb9fb9e91 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:22.023017 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9976a-3a37-42bc-8202-54adb9fb9e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:22.035593 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:22.095484 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-b0b9976a-3a37-42bc-8202-54adb9fb9e91 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:59:22.096727 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b0b9976a-3a37-42bc-8202-54adb9fb9e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:22.201395 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 690/1381] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:59:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:59:22.494668 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd71cc62-0ead-4187-8c6c-4a7302fcf1c3 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:59:22.496239 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-cd71cc62-0ead-4187-8c6c-4a7302fcf1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:22.651064 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:22.663302 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd71cc62-0ead-4187-8c6c-4a7302fcf1c3 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:59:22.785850 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9102a81-f3a5-47b8-8f5a-eb6e71d40bb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:59:22.786469 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9102a81-f3a5-47b8-8f5a-eb6e71d40bb8 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:22.786747 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9102a81-f3a5-47b8-8f5a-eb6e71d40bb8 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:22.787018 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9102a81-f3a5-47b8-8f5a-eb6e71d40bb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:22.788689 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d9102a81-f3a5-47b8-8f5a-eb6e71d40bb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:22.882241 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9102a81-f3a5-47b8-8f5a-eb6e71d40bb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:59:22.883838 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9102a81-f3a5-47b8-8f5a-eb6e71d40bb8 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:59:22.895374 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:22.904487 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 692/1382] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 00:59:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 00:59:25.171452 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed4439-d3a6-47a4-8177-ca4c400dd9e4 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:25.172027 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed4439-d3a6-47a4-8177-ca4c400dd9e4 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:25.172150 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed4439-d3a6-47a4-8177-ca4c400dd9e4 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:25.173942 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 691/1383] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:59:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:59:25.180102 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba6005-d66f-41f6-9f75-4aee48d8e01e None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:25.180573 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba6005-d66f-41f6-9f75-4aee48d8e01e None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:25.180680 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba6005-d66f-41f6-9f75-4aee48d8e01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:25.251998 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-cbba6005-d66f-41f6-9f75-4aee48d8e01e None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:59:25.253316 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-cbba6005-d66f-41f6-9f75-4aee48d8e01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:25.355350 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 693/1384] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:59:25.722048 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ed4439-d3a6-47a4-8177-ca4c400dd9e4 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:59:25.724017 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b3ed4439-d3a6-47a4-8177-ca4c400dd9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:25.831758 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:25.904151 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ed4439-d3a6-47a4-8177-ca4c400dd9e4 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:59:26.038313 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55297889-904b-4334-baea-cb1879b43886 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:59:26.038926 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55297889-904b-4334-baea-cb1879b43886 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:26.039209 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55297889-904b-4334-baea-cb1879b43886 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:26.039489 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55297889-904b-4334-baea-cb1879b43886 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:26.041117 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-55297889-904b-4334-baea-cb1879b43886 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:26.094627 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:26.134014 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55297889-904b-4334-baea-cb1879b43886 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:59:26.135627 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55297889-904b-4334-baea-cb1879b43886 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:59:26.154930 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 692/1385] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 00:59:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 00:59:28.113161 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafd739-8fdb-4042-adc6-d6b8a3ca0a0b None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:28.113701 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafd739-8fdb-4042-adc6-d6b8a3ca0a0b None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:28.113701 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafd739-8fdb-4042-adc6-d6b8a3ca0a0b None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:28.115117 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 694/1386] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:59:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:59:28.120331 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9be9c-2430-4361-9f9b-3891a6d83de8 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:28.120747 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9be9c-2430-4361-9f9b-3891a6d83de8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:28.120909 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9be9c-2430-4361-9f9b-3891a6d83de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:28.187022 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-00c9be9c-2430-4361-9f9b-3891a6d83de8 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:59:28.188699 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-00c9be9c-2430-4361-9f9b-3891a6d83de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:28.311425 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 693/1387] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:59:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:59:28.670539 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbafd739-8fdb-4042-adc6-d6b8a3ca0a0b None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:59:28.672151 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-bbafd739-8fdb-4042-adc6-d6b8a3ca0a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:28.754083 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:28.839878 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbafd739-8fdb-4042-adc6-d6b8a3ca0a0b None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:59:28.964477 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2894e-f40d-4af6-8fb4-f9751db9cabc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:59:28.965122 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2894e-f40d-4af6-8fb4-f9751db9cabc service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:28.965358 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2894e-f40d-4af6-8fb4-f9751db9cabc service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:28.965630 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2894e-f40d-4af6-8fb4-f9751db9cabc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:28.967971 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-06d2894e-f40d-4af6-8fb4-f9751db9cabc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:29.007262 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:29.061764 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2894e-f40d-4af6-8fb4-f9751db9cabc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:59:29.063506 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2894e-f40d-4af6-8fb4-f9751db9cabc service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:59:29.085911 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 695/1388] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 00:59:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 00:59:31.138340 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727862a-871f-437b-a50a-04e7480718ea None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:31.138929 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727862a-871f-437b-a50a-04e7480718ea None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:31.138929 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727862a-871f-437b-a50a-04e7480718ea None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:31.140219 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 694/1389] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:59:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:59:31.143860 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0066a-943d-4994-889e-69124be43bc4 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:31.144273 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0066a-943d-4994-889e-69124be43bc4 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:31.144422 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0066a-943d-4994-889e-69124be43bc4 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:31.145031 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc1e1a-65ef-4ace-9fff-07429ea55f7a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:31.145313 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc1e1a-65ef-4ace-9fff-07429ea55f7a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:31.145589 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc1e1a-65ef-4ace-9fff-07429ea55f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:31.145903 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 696/1390] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:59:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:59:31.151134 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f1db0-dd85-4103-b457-7fdf67f069a8 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:31.151515 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f1db0-dd85-4103-b457-7fdf67f069a8 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:31.151797 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f1db0-dd85-4103-b457-7fdf67f069a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:31.181164 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:31.212658 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-4ecc1e1a-65ef-4ace-9fff-07429ea55f7a None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:59:31.213874 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4ecc1e1a-65ef-4ace-9fff-07429ea55f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:31.219017 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-076f1db0-dd85-4103-b457-7fdf67f069a8 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:59:31.222709 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-076f1db0-dd85-4103-b457-7fdf67f069a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:31.315191 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 695/1391] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 00:59:31.321108 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 697/1392] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 00:59:31.611112 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc1e1a-65ef-4ace-9fff-07429ea55f7a None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:59:31.612383 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4ecc1e1a-65ef-4ace-9fff-07429ea55f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:31.614110 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6a690-6b28-4a67-9756-8c93033d5fdb None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:31.614395 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6a690-6b28-4a67-9756-8c93033d5fdb None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:31.614675 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6a690-6b28-4a67-9756-8c93033d5fdb None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:31.616010 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 698/1393] 10.222.0.27 () {54 vars in 961 bytes} [Sun Sep 4 00:59:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 00:59:31.620888 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb902e0-862c-4d08-87cd-ff99870920d4 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:31.621195 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb902e0-862c-4d08-87cd-ff99870920d4 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:31.621437 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb902e0-862c-4d08-87cd-ff99870920d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:31.639301 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:31.703599 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-cbb902e0-862c-4d08-87cd-ff99870920d4 None None] MFA Rules not processed for user `1179503bdab547c99175a27b7ce27b9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:59:31.704998 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-cbb902e0-862c-4d08-87cd-ff99870920d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:31.721450 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:31.729606 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc1e1a-65ef-4ace-9fff-07429ea55f7a None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:59:31.783811 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 699/1394] 10.222.0.27 () {58 vars in 1058 bytes} [Sun Sep 4 00:59:31 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 00:59:31.791531 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb902e0-862c-4d08-87cd-ff99870920d4 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:59:31.799181 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb902e0-862c-4d08-87cd-ff99870920d4 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:59:31.849800 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc361227-305c-4aaf-84f5-091e5fc03423 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:59:31.850324 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc361227-305c-4aaf-84f5-091e5fc03423 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:31.850605 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc361227-305c-4aaf-84f5-091e5fc03423 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:31.850897 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc361227-305c-4aaf-84f5-091e5fc03423 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:31.852505 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bc361227-305c-4aaf-84f5-091e5fc03423 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:31.876152 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c256abf2-f8e7-4786-a51a-64f2ea73ba26 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1179503bdab547c99175a27b7ce27b9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:59:31.876797 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256abf2-f8e7-4786-a51a-64f2ea73ba26 service glance] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:31.877085 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256abf2-f8e7-4786-a51a-64f2ea73ba26 service glance] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:31.877357 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256abf2-f8e7-4786-a51a-64f2ea73ba26 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:31.878960 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c256abf2-f8e7-4786-a51a-64f2ea73ba26 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:31.909592 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:31.947160 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc361227-305c-4aaf-84f5-091e5fc03423 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:59:31.948831 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc361227-305c-4aaf-84f5-091e5fc03423 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:59:31.964858 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:31.967903 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 696/1395] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 00:59:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 00:59:31.991174 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c256abf2-f8e7-4786-a51a-64f2ea73ba26 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:59:31.992679 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c256abf2-f8e7-4786-a51a-64f2ea73ba26 service glance] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:59:32.011605 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 700/1396] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 00:59:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 00:59:32.893707 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5e841-597b-491b-8857-c96284cf1776 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:32.894202 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5e841-597b-491b-8857-c96284cf1776 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:32.894439 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5e841-597b-491b-8857-c96284cf1776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:32.960745 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-d5f5e841-597b-491b-8857-c96284cf1776 None None] MFA Rules not processed for user `d6d01ca673a1404c9b4cea51036c595c`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:59:32.961941 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d5f5e841-597b-491b-8857-c96284cf1776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:33.049407 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 697/1397] 10.222.0.27 () {58 vars in 1009 bytes} [Sun Sep 4 00:59:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 00:59:33.095710 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7a141-f526-4e61-8d0e-8ccc4c51523d None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:33.096180 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7a141-f526-4e61-8d0e-8ccc4c51523d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:33.096572 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7a141-f526-4e61-8d0e-8ccc4c51523d None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:33.098044 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 701/1398] 10.222.0.27 () {54 vars in 943 bytes} [Sun Sep 4 00:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:59:33.104542 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7e063-f71a-4b87-a6dd-d28e713b2e61 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:33.104832 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7e063-f71a-4b87-a6dd-d28e713b2e61 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:33.105106 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7e063-f71a-4b87-a6dd-d28e713b2e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:33.180720 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-ecb7e063-f71a-4b87-a6dd-d28e713b2e61 None None] MFA Rules not processed for user `2297d8b3fe8e47eb9fd39a6e3eea8f54`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:59:33.181922 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ecb7e063-f71a-4b87-a6dd-d28e713b2e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:33.216594 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:33.295737 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 698/1399] 10.222.0.27 () {58 vars in 1040 bytes} [Sun Sep 4 00:59:33 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 00:59:33.305407 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7a141-f526-4e61-8d0e-8ccc4c51523d None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:59:33.307009 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-7fe7a141-f526-4e61-8d0e-8ccc4c51523d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:33.439402 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:33.441981 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7a141-f526-4e61-8d0e-8ccc4c51523d None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:59:33.548886 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc6f6a8-f745-46cd-b553-d6237e864f78 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2297d8b3fe8e47eb9fd39a6e3eea8f54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:59:33.549492 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6f6a8-f745-46cd-b553-d6237e864f78 service swift] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:33.549768 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6f6a8-f745-46cd-b553-d6237e864f78 service swift] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:33.550035 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6f6a8-f745-46cd-b553-d6237e864f78 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:33.551655 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8dc6f6a8-f745-46cd-b553-d6237e864f78 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:33.634043 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc6f6a8-f745-46cd-b553-d6237e864f78 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:59:33.635649 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc6f6a8-f745-46cd-b553-d6237e864f78 service swift] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:59:33.644034 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 702/1400] 10.222.0.27 () {60 vars in 1384 bytes} [Sun Sep 4 00:59:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 04 00:59:34.070921 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9495f59-1b76-4307-bfb1-947372774aef None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:34.071462 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9495f59-1b76-4307-bfb1-947372774aef None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:34.071462 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9495f59-1b76-4307-bfb1-947372774aef None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:34.076819 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 699/1401] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:59:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:59:34.079504 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a1792-1588-4f24-9093-b323840e5371 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:34.079909 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a1792-1588-4f24-9093-b323840e5371 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:34.080246 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a1792-1588-4f24-9093-b323840e5371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:34.121672 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:34.156407 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-ee1a1792-1588-4f24-9093-b323840e5371 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:59:34.158172 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ee1a1792-1588-4f24-9093-b323840e5371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:34.265472 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 703/1402] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:59:34.562412 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9495f59-1b76-4307-bfb1-947372774aef None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:59:34.563926 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e9495f59-1b76-4307-bfb1-947372774aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:34.728410 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9495f59-1b76-4307-bfb1-947372774aef None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:59:34.732328 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:34.852542 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04cae64-cc33-4c43-ae2f-6b339cdca8e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:59:34.853476 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cae64-cc33-4c43-ae2f-6b339cdca8e1 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:34.853754 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cae64-cc33-4c43-ae2f-6b339cdca8e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:34.854026 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cae64-cc33-4c43-ae2f-6b339cdca8e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:34.855685 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d04cae64-cc33-4c43-ae2f-6b339cdca8e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:34.959263 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04cae64-cc33-4c43-ae2f-6b339cdca8e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:59:34.961082 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04cae64-cc33-4c43-ae2f-6b339cdca8e1 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:59:34.980361 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 700/1403] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 00:59:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 00:59:37.093939 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a32b8-bfc2-4f53-b620-33eb5a947ff6 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:37.094518 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a32b8-bfc2-4f53-b620-33eb5a947ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:37.094518 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a32b8-bfc2-4f53-b620-33eb5a947ff6 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:37.095831 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 704/1404] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:59:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:59:37.101248 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158bdde-b680-475d-bcf4-54e241062b55 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:37.101653 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158bdde-b680-475d-bcf4-54e241062b55 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:37.101859 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158bdde-b680-475d-bcf4-54e241062b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:37.115175 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:37.179001 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-d158bdde-b680-475d-bcf4-54e241062b55 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:59:37.180476 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d158bdde-b680-475d-bcf4-54e241062b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:37.292346 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 701/1405] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:59:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:59:37.588634 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702a32b8-bfc2-4f53-b620-33eb5a947ff6 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:59:37.590181 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-702a32b8-bfc2-4f53-b620-33eb5a947ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:37.705465 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:37.752587 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702a32b8-bfc2-4f53-b620-33eb5a947ff6 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:59:37.897736 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473725b-bae2-4df0-b77b-2c515226f9df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:59:37.898370 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473725b-bae2-4df0-b77b-2c515226f9df service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:37.898645 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473725b-bae2-4df0-b77b-2c515226f9df service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:37.898923 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473725b-bae2-4df0-b77b-2c515226f9df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:37.904519 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e473725b-bae2-4df0-b77b-2c515226f9df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:37.994759 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:38.018711 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473725b-bae2-4df0-b77b-2c515226f9df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:59:38.020407 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473725b-bae2-4df0-b77b-2c515226f9df service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:59:38.041777 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 705/1406] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 00:59:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 00:59:40.208744 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82cfd6-a728-4bd7-ae9c-b2e4cc2b5188 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:40.209324 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82cfd6-a728-4bd7-ae9c-b2e4cc2b5188 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:40.209324 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82cfd6-a728-4bd7-ae9c-b2e4cc2b5188 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:40.210903 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 702/1407] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 00:59:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:59:40.216966 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5badb-9da3-4b8a-8fa8-b3a574e28822 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:40.217753 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5badb-9da3-4b8a-8fa8-b3a574e28822 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:40.217753 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5badb-9da3-4b8a-8fa8-b3a574e28822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:40.330876 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-06d5badb-9da3-4b8a-8fa8-b3a574e28822 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:59:40.331664 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-06d5badb-9da3-4b8a-8fa8-b3a574e28822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:40.468174 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 706/1408] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 00:59:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:59:40.768625 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82cfd6-a728-4bd7-ae9c-b2e4cc2b5188 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:59:40.770223 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-cb82cfd6-a728-4bd7-ae9c-b2e4cc2b5188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:40.876410 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:40.956809 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82cfd6-a728-4bd7-ae9c-b2e4cc2b5188 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 00:59:41.104215 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c61ed-a628-4815-b907-cf873b807fb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 00:59:41.105026 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c61ed-a628-4815-b907-cf873b807fb6 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:41.105372 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c61ed-a628-4815-b907-cf873b807fb6 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:41.105752 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c61ed-a628-4815-b907-cf873b807fb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:41.107517 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-302c61ed-a628-4815-b907-cf873b807fb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:41.163418 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:41.207369 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302c61ed-a628-4815-b907-cf873b807fb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 00:59:41.209006 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302c61ed-a628-4815-b907-cf873b807fb6 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 00:59:41.228869 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 703/1409] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 00:59:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 00:59:50.464168 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=79 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:51.233347 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=23 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:56.712536 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e40172-453c-46f2-bd05-8ca8811735f7 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:56.713079 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e40172-453c-46f2-bd05-8ca8811735f7 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:56.713176 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e40172-453c-46f2-bd05-8ca8811735f7 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:56.714459 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 707/1410] 10.222.0.27 () {54 vars in 918 bytes} [Sun Sep 4 00:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:59:56.720048 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e76f8-cdd8-4ab6-aec5-ab6a23b7340a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:56.720517 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e76f8-cdd8-4ab6-aec5-ab6a23b7340a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:56.720633 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e76f8-cdd8-4ab6-aec5-ab6a23b7340a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:56.796501 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-fa8e76f8-cdd8-4ab6-aec5-ab6a23b7340a None None] MFA Rules not processed for user `018802a8d9c041119ad708430f885a89`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:59:56.797715 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-fa8e76f8-cdd8-4ab6-aec5-ab6a23b7340a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:56.903112 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 704/1411] 10.222.0.27 () {58 vars in 1015 bytes} [Sun Sep 4 00:59:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:59:56.916959 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a79886-7a05-4a61-b5e9-626c2f19d0f9 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:56.917251 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a79886-7a05-4a61-b5e9-626c2f19d0f9 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:56.917537 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a79886-7a05-4a61-b5e9-626c2f19d0f9 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:56.919411 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 708/1412] 10.222.0.27 () {54 vars in 918 bytes} [Sun Sep 4 00:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:59:56.923986 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323faf0c-a0d6-4cda-a0b3-0b33350aa2fc None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:56.924272 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323faf0c-a0d6-4cda-a0b3-0b33350aa2fc None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:56.924581 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323faf0c-a0d6-4cda-a0b3-0b33350aa2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:56.998810 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:57.001345 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-323faf0c-a0d6-4cda-a0b3-0b33350aa2fc None None] MFA Rules not processed for user `68bc00bda5194cd8a7928f8c4df4ccd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:59:57.002539 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-323faf0c-a0d6-4cda-a0b3-0b33350aa2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:57.118095 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 705/1413] 10.222.0.27 () {58 vars in 1015 bytes} [Sun Sep 4 00:59:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:59:59.489509 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8b9c2-e441-4e35-bf9a-a3460cf1a736 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:59.490043 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8b9c2-e441-4e35-bf9a-a3460cf1a736 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:59.490043 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8b9c2-e441-4e35-bf9a-a3460cf1a736 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:59.491399 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 4 00:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 00:59:59.497198 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef56bc-5af2-462c-8c5e-41fd0a3267e2 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 00:59:59.497638 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef56bc-5af2-462c-8c5e-41fd0a3267e2 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 00:59:59.497746 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef56bc-5af2-462c-8c5e-41fd0a3267e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 00:59:59.564181 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-22ef56bc-5af2-462c-8c5e-41fd0a3267e2 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 00:59:59.565417 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-22ef56bc-5af2-462c-8c5e-41fd0a3267e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 00:59:59.621856 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:59.663098 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 4 00:59:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 00:59:59.957076 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed8b9c2-e441-4e35-bf9a-a3460cf1a736 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 00:59:59.958465 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-9ed8b9c2-e441-4e35-bf9a-a3460cf1a736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:00.100599 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed8b9c2-e441-4e35-bf9a-a3460cf1a736 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:00:00.172049 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:00.211528 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60837049-2713-404c-9dff-300c47988a24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:00:00.212059 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60837049-2713-404c-9dff-300c47988a24 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:00.212340 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60837049-2713-404c-9dff-300c47988a24 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:00.212637 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60837049-2713-404c-9dff-300c47988a24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:00.214224 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-60837049-2713-404c-9dff-300c47988a24 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:00.304195 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60837049-2713-404c-9dff-300c47988a24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:00:00.305934 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60837049-2713-404c-9dff-300c47988a24 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:00:00.323597 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 710/1416] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 00:59:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:00:01.227297 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0c1bf-f418-4756-973d-6f004e3fb8ac None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:01.227824 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0c1bf-f418-4756-973d-6f004e3fb8ac None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:01.227824 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0c1bf-f418-4756-973d-6f004e3fb8ac None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:01.229236 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 4 01:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:00:01.234404 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3efbb-0eb6-4c53-bd1d-ff7ea15de6db None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:01.234827 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3efbb-0eb6-4c53-bd1d-ff7ea15de6db None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:01.235001 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3efbb-0eb6-4c53-bd1d-ff7ea15de6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:01.308876 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-3be3efbb-0eb6-4c53-bd1d-ff7ea15de6db None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:00:01.310080 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3be3efbb-0eb6-4c53-bd1d-ff7ea15de6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:01.336154 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:01.405289 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 4 01:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:00:01.555940 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac0c1bf-f418-4756-973d-6f004e3fb8ac None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:00:01.557521 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6ac0c1bf-f418-4756-973d-6f004e3fb8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:01.705002 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac0c1bf-f418-4756-973d-6f004e3fb8ac None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:00:01.744468 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:01.819165 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc4d16-9b17-4961-8876-a563254355f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:00:01.819769 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc4d16-9b17-4961-8876-a563254355f6 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:01.820066 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc4d16-9b17-4961-8876-a563254355f6 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:01.820320 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc4d16-9b17-4961-8876-a563254355f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:01.821947 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3acc4d16-9b17-4961-8876-a563254355f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:01.913559 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc4d16-9b17-4961-8876-a563254355f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:00:01.915129 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc4d16-9b17-4961-8876-a563254355f6 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:00:01.934654 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 708/1419] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:00:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:00:02.811651 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be64108-85f8-4890-9abb-7c11a9aceb5c None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:02.812166 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be64108-85f8-4890-9abb-7c11a9aceb5c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:02.812166 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be64108-85f8-4890-9abb-7c11a9aceb5c None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:02.813477 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 4 01:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:00:02.818263 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b52d66-8990-4d38-bced-9c1f95b1b33c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:02.818729 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b52d66-8990-4d38-bced-9c1f95b1b33c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:02.818876 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b52d66-8990-4d38-bced-9c1f95b1b33c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:02.873480 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:02.897259 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-56b52d66-8990-4d38-bced-9c1f95b1b33c None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:00:02.898477 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-56b52d66-8990-4d38-bced-9c1f95b1b33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:02.994048 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 4 01:00:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:00:03.147175 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be64108-85f8-4890-9abb-7c11a9aceb5c None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:00:03.148645 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2be64108-85f8-4890-9abb-7c11a9aceb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:03.294125 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be64108-85f8-4890-9abb-7c11a9aceb5c None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:00:03.306352 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:03.408024 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8d8d6-6a3e-4a89-a634-3e88a6b6a458 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:00:03.408578 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8d8d6-6a3e-4a89-a634-3e88a6b6a458 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:03.408864 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8d8d6-6a3e-4a89-a634-3e88a6b6a458 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:03.409136 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8d8d6-6a3e-4a89-a634-3e88a6b6a458 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:03.410730 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-04b8d8d6-6a3e-4a89-a634-3e88a6b6a458 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:03.500111 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b8d8d6-6a3e-4a89-a634-3e88a6b6a458 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:00:03.501744 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b8d8d6-6a3e-4a89-a634-3e88a6b6a458 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:00:03.519710 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 713/1422] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:00:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:00:04.373050 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd275ea2-bb35-4859-8b02-b4d87a63c120 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:04.373579 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd275ea2-bb35-4859-8b02-b4d87a63c120 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:04.373579 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd275ea2-bb35-4859-8b02-b4d87a63c120 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:04.375007 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 4 01:00:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:00:04.379770 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d2025-008d-402b-a9b4-40046d4b1dd1 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:04.380186 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d2025-008d-402b-a9b4-40046d4b1dd1 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:04.380299 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d2025-008d-402b-a9b4-40046d4b1dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:04.400146 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:04.455557 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-b65d2025-008d-402b-a9b4-40046d4b1dd1 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:00:04.456839 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b65d2025-008d-402b-a9b4-40046d4b1dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:04.553751 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 4 01:00:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:00:04.702202 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd275ea2-bb35-4859-8b02-b4d87a63c120 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:00:04.703709 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-dd275ea2-bb35-4859-8b02-b4d87a63c120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:04.811515 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:04.847185 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd275ea2-bb35-4859-8b02-b4d87a63c120 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:00:04.961026 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5bce62-fd90-4f25-85e0-a5afdbeb8967 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:00:04.965922 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5bce62-fd90-4f25-85e0-a5afdbeb8967 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:04.965922 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5bce62-fd90-4f25-85e0-a5afdbeb8967 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:04.965922 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5bce62-fd90-4f25-85e0-a5afdbeb8967 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:04.965922 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5b5bce62-fd90-4f25-85e0-a5afdbeb8967 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:05.043178 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:05.056116 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5bce62-fd90-4f25-85e0-a5afdbeb8967 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:00:05.057730 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5bce62-fd90-4f25-85e0-a5afdbeb8967 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:00:05.077622 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 711/1425] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:00:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:00:05.939881 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8350fa-82e7-47a9-8ae4-e374e1feceed None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:05.940425 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8350fa-82e7-47a9-8ae4-e374e1feceed None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:05.940425 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8350fa-82e7-47a9-8ae4-e374e1feceed None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:05.941712 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 4 01:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:00:05.946659 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988222a8-38d5-4c75-8a3f-a9aa77000009 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:05.947078 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988222a8-38d5-4c75-8a3f-a9aa77000009 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:05.947202 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988222a8-38d5-4c75-8a3f-a9aa77000009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:06.024870 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-988222a8-38d5-4c75-8a3f-a9aa77000009 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:00:06.026071 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-988222a8-38d5-4c75-8a3f-a9aa77000009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:06.123357 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 4 01:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:00:06.276503 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8350fa-82e7-47a9-8ae4-e374e1feceed None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:00:06.278756 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-fa8350fa-82e7-47a9-8ae4-e374e1feceed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:06.361787 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:06.422081 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8350fa-82e7-47a9-8ae4-e374e1feceed None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:00:06.532864 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b121d-ea1c-4309-be45-35af294889fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:00:06.533463 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b121d-ea1c-4309-be45-35af294889fb service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:06.533737 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b121d-ea1c-4309-be45-35af294889fb service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:06.534006 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b121d-ea1c-4309-be45-35af294889fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:06.535637 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d36b121d-ea1c-4309-be45-35af294889fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:06.582227 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:06.623430 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b121d-ea1c-4309-be45-35af294889fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:00:06.625049 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b121d-ea1c-4309-be45-35af294889fb service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:00:06.643505 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 716/1428] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:00:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:00:07.499646 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3643820-2aff-4bf1-8a76-351767086b24 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:07.500199 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3643820-2aff-4bf1-8a76-351767086b24 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:07.500199 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3643820-2aff-4bf1-8a76-351767086b24 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:07.501569 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 4 01:00:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:00:07.506504 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bdd7d8-e1e8-4320-b868-9a21be6cb0ad None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:07.506938 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bdd7d8-e1e8-4320-b868-9a21be6cb0ad None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:07.507050 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bdd7d8-e1e8-4320-b868-9a21be6cb0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:07.582987 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-50bdd7d8-e1e8-4320-b868-9a21be6cb0ad None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:00:07.584241 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-50bdd7d8-e1e8-4320-b868-9a21be6cb0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:07.681124 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 4 01:00:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:00:07.829982 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3643820-2aff-4bf1-8a76-351767086b24 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:00:07.831399 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c3643820-2aff-4bf1-8a76-351767086b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:07.881107 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:07.980547 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3643820-2aff-4bf1-8a76-351767086b24 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:00:08.092010 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ca848-0c9a-458d-95c8-189927301db2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:00:08.092593 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ca848-0c9a-458d-95c8-189927301db2 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:08.092875 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ca848-0c9a-458d-95c8-189927301db2 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:08.093161 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ca848-0c9a-458d-95c8-189927301db2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:08.094750 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5c8ca848-0c9a-458d-95c8-189927301db2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:08.101710 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:08.184885 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8ca848-0c9a-458d-95c8-189927301db2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:00:08.186451 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8ca848-0c9a-458d-95c8-189927301db2 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:00:08.204019 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 714/1431] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:00:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:00:09.045806 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e0d22-f90a-41aa-a995-33602777cdd1 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:09.046377 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e0d22-f90a-41aa-a995-33602777cdd1 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:09.046470 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e0d22-f90a-41aa-a995-33602777cdd1 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:09.047791 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 4 01:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:00:09.052941 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5297cfa-1686-4955-adb0-8c78c811ffbe None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:09.053439 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5297cfa-1686-4955-adb0-8c78c811ffbe None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:09.053586 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5297cfa-1686-4955-adb0-8c78c811ffbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:09.131472 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-e5297cfa-1686-4955-adb0-8c78c811ffbe None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:00:09.132763 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e5297cfa-1686-4955-adb0-8c78c811ffbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:09.201405 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:09.230403 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 4 01:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:00:09.379722 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591e0d22-f90a-41aa-a995-33602777cdd1 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:00:09.381189 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-591e0d22-f90a-41aa-a995-33602777cdd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:09.399189 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:09.525237 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591e0d22-f90a-41aa-a995-33602777cdd1 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:00:09.611469 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:09.636478 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98002869-d633-4cc3-bc56-598faa3df6b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:00:09.637022 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98002869-d633-4cc3-bc56-598faa3df6b4 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:09.637300 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98002869-d633-4cc3-bc56-598faa3df6b4 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:09.637566 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98002869-d633-4cc3-bc56-598faa3df6b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:09.639150 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-98002869-d633-4cc3-bc56-598faa3df6b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:09.728016 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98002869-d633-4cc3-bc56-598faa3df6b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:00:09.729638 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98002869-d633-4cc3-bc56-598faa3df6b4 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:00:09.748000 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 719/1434] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:00:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:00:10.591520 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41b17d-7245-466d-ab77-895a9a019f95 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:10.592070 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41b17d-7245-466d-ab77-895a9a019f95 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:10.592070 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41b17d-7245-466d-ab77-895a9a019f95 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:10.593473 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 4 01:00:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:00:10.598869 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacdb8c5-f285-467b-b9aa-d594ddaa482e None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:10.598869 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacdb8c5-f285-467b-b9aa-d594ddaa482e None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:10.599367 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacdb8c5-f285-467b-b9aa-d594ddaa482e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:10.676127 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-cacdb8c5-f285-467b-b9aa-d594ddaa482e None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:00:10.677415 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-cacdb8c5-f285-467b-b9aa-d594ddaa482e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:10.719339 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:10.774854 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 4 01:00:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:00:10.925626 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41b17d-7245-466d-ab77-895a9a019f95 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:00:10.927052 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2f41b17d-7245-466d-ab77-895a9a019f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:11.071639 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41b17d-7245-466d-ab77-895a9a019f95 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:00:11.127331 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:11.187999 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5540e-2d22-45b8-b062-f218fc16e3df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:00:11.188551 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5540e-2d22-45b8-b062-f218fc16e3df service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:11.188831 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5540e-2d22-45b8-b062-f218fc16e3df service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:11.189102 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5540e-2d22-45b8-b062-f218fc16e3df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:11.190680 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b9e5540e-2d22-45b8-b062-f218fc16e3df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:11.279752 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e5540e-2d22-45b8-b062-f218fc16e3df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:00:11.281371 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e5540e-2d22-45b8-b062-f218fc16e3df service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:00:11.299562 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 717/1437] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:00:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:00:12.155121 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f906c03-0e25-44c2-a0cb-5fadcfc06576 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:12.155696 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f906c03-0e25-44c2-a0cb-5fadcfc06576 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:12.155696 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f906c03-0e25-44c2-a0cb-5fadcfc06576 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:12.157018 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 4 01:00:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:00:12.162003 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb019be9-cc0c-4638-8a7b-20b9c177423f None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:12.162441 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb019be9-cc0c-4638-8a7b-20b9c177423f None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:12.162554 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb019be9-cc0c-4638-8a7b-20b9c177423f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:12.239358 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-fb019be9-cc0c-4638-8a7b-20b9c177423f None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:00:12.240611 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-fb019be9-cc0c-4638-8a7b-20b9c177423f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:12.242633 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:12.338618 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 4 01:00:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:00:12.494207 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f906c03-0e25-44c2-a0cb-5fadcfc06576 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:00:12.495776 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-1f906c03-0e25-44c2-a0cb-5fadcfc06576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:12.641048 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f906c03-0e25-44c2-a0cb-5fadcfc06576 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:00:12.669837 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:12.754235 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34323d6f-4747-4609-a49f-8a1b4f45e000 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:00:12.754996 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34323d6f-4747-4609-a49f-8a1b4f45e000 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:12.755273 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34323d6f-4747-4609-a49f-8a1b4f45e000 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:12.755540 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34323d6f-4747-4609-a49f-8a1b4f45e000 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:12.757173 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-34323d6f-4747-4609-a49f-8a1b4f45e000 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:12.845303 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34323d6f-4747-4609-a49f-8a1b4f45e000 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:00:12.847095 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34323d6f-4747-4609-a49f-8a1b4f45e000 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:00:12.864746 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 722/1440] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:00:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:00:14.694805 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667ce98-a2b5-4843-91c0-37c84170c038 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:14.695346 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667ce98-a2b5-4843-91c0-37c84170c038 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:14.695471 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667ce98-a2b5-4843-91c0-37c84170c038 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:14.696774 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 4 01:00:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:00:14.702201 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b4163-c2db-4575-af92-fb1870d91415 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:14.702585 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b4163-c2db-4575-af92-fb1870d91415 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:14.702743 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b4163-c2db-4575-af92-fb1870d91415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:14.743756 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:14.769632 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-c52b4163-c2db-4575-af92-fb1870d91415 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:00:14.770865 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c52b4163-c2db-4575-af92-fb1870d91415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:14.866635 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 4 01:00:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:00:15.151342 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667ce98-a2b5-4843-91c0-37c84170c038 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:00:15.153620 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0667ce98-a2b5-4843-91c0-37c84170c038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:15.257913 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667ce98-a2b5-4843-91c0-37c84170c038 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:00:15.290657 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:15.332876 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f9cb42-4702-4ca1-8b5c-77d1a424cf72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:00:15.333486 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9cb42-4702-4ca1-8b5c-77d1a424cf72 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:15.333771 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9cb42-4702-4ca1-8b5c-77d1a424cf72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:15.334050 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9cb42-4702-4ca1-8b5c-77d1a424cf72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:15.335675 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-34f9cb42-4702-4ca1-8b5c-77d1a424cf72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:15.443881 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f9cb42-4702-4ca1-8b5c-77d1a424cf72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:00:15.445266 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f9cb42-4702-4ca1-8b5c-77d1a424cf72 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:00:15.463838 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 720/1443] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:00:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:00:17.572436 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c27f5-79c2-4b8f-8da7-04ab60128336 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:17.572990 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c27f5-79c2-4b8f-8da7-04ab60128336 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:17.573116 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c27f5-79c2-4b8f-8da7-04ab60128336 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:17.574892 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 4 01:00:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:00:17.580373 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f016e0d-d505-430c-bba5-ecb83bfea4df None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:17.580836 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f016e0d-d505-430c-bba5-ecb83bfea4df None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:17.580935 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f016e0d-d505-430c-bba5-ecb83bfea4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:17.646032 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-8f016e0d-d505-430c-bba5-ecb83bfea4df None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:00:17.647250 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8f016e0d-d505-430c-bba5-ecb83bfea4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:17.660184 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:17.744875 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 4 01:00:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:00:18.046433 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c27f5-79c2-4b8f-8da7-04ab60128336 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:00:18.047985 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-7e4c27f5-79c2-4b8f-8da7-04ab60128336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:18.156080 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c27f5-79c2-4b8f-8da7-04ab60128336 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:00:18.168740 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:18.231685 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ebbe0-0dce-4185-bc30-800f7f0ced45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:00:18.232299 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ebbe0-0dce-4185-bc30-800f7f0ced45 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:18.232622 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ebbe0-0dce-4185-bc30-800f7f0ced45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:18.232920 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ebbe0-0dce-4185-bc30-800f7f0ced45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:18.234538 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-615ebbe0-0dce-4185-bc30-800f7f0ced45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:18.345218 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615ebbe0-0dce-4185-bc30-800f7f0ced45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:00:18.346753 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615ebbe0-0dce-4185-bc30-800f7f0ced45 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:00:18.366790 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 725/1446] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:00:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 01:00:20.645908 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c63f63-df6a-4bbf-a625-461fc2708b93 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:20.646444 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c63f63-df6a-4bbf-a625-461fc2708b93 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:20.646561 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c63f63-df6a-4bbf-a625-461fc2708b93 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:20.647849 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 4 01:00:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:00:20.654315 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0abf19e-42aa-4dcc-8801-4c8dcd189802 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:20.654753 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0abf19e-42aa-4dcc-8801-4c8dcd189802 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:20.654853 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0abf19e-42aa-4dcc-8801-4c8dcd189802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:20.701473 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:20.720598 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-b0abf19e-42aa-4dcc-8801-4c8dcd189802 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:00:20.724585 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b0abf19e-42aa-4dcc-8801-4c8dcd189802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:20.816810 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 4 01:00:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:00:21.104745 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c63f63-df6a-4bbf-a625-461fc2708b93 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:00:21.106288 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-16c63f63-df6a-4bbf-a625-461fc2708b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:21.210651 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c63f63-df6a-4bbf-a625-461fc2708b93 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:00:21.251646 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:21.286071 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a652e174-bd29-4578-8a14-1595f94eb63f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:00:21.286672 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652e174-bd29-4578-8a14-1595f94eb63f service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:21.286947 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652e174-bd29-4578-8a14-1595f94eb63f service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:21.287215 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652e174-bd29-4578-8a14-1595f94eb63f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:21.288941 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a652e174-bd29-4578-8a14-1595f94eb63f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:21.397474 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a652e174-bd29-4578-8a14-1595f94eb63f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:00:21.399070 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a652e174-bd29-4578-8a14-1595f94eb63f service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:00:21.417657 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 723/1449] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:00:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:00:23.473505 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5125446-d166-45cd-ad22-7a33c823f0df None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:23.474056 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5125446-d166-45cd-ad22-7a33c823f0df None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:23.474056 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5125446-d166-45cd-ad22-7a33c823f0df None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:23.475392 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 4 01:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:00:23.489272 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18faffec-dd7e-4363-9e3e-d972a9716b96 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:23.489272 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18faffec-dd7e-4363-9e3e-d972a9716b96 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:23.489272 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18faffec-dd7e-4363-9e3e-d972a9716b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:23.548608 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-18faffec-dd7e-4363-9e3e-d972a9716b96 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:00:23.549801 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-18faffec-dd7e-4363-9e3e-d972a9716b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:23.569742 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:23.645530 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 4 01:00:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:00:23.927446 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5125446-d166-45cd-ad22-7a33c823f0df None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:00:23.929014 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a5125446-d166-45cd-ad22-7a33c823f0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:24.034285 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5125446-d166-45cd-ad22-7a33c823f0df None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:00:24.054214 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:24.109766 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603a00d-e840-4462-bdd2-cdcd213a3526 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:00:24.110382 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603a00d-e840-4462-bdd2-cdcd213a3526 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:24.110668 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603a00d-e840-4462-bdd2-cdcd213a3526 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:24.110946 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603a00d-e840-4462-bdd2-cdcd213a3526 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:24.112587 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a603a00d-e840-4462-bdd2-cdcd213a3526 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:24.224069 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603a00d-e840-4462-bdd2-cdcd213a3526 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:00:24.225485 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603a00d-e840-4462-bdd2-cdcd213a3526 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:00:24.244149 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 728/1452] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:00:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:00:26.217554 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a095a-edb4-4dcf-846d-4740cf39baa2 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:26.218168 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a095a-edb4-4dcf-846d-4740cf39baa2 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:26.218168 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a095a-edb4-4dcf-846d-4740cf39baa2 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:26.219540 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 4 01:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:00:26.225222 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd0a7f-e84f-436e-81d9-ff6296b41947 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:26.227165 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd0a7f-e84f-436e-81d9-ff6296b41947 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:26.227165 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd0a7f-e84f-436e-81d9-ff6296b41947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:26.291042 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:26.293999 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-3ccd0a7f-e84f-436e-81d9-ff6296b41947 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:00:26.295209 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3ccd0a7f-e84f-436e-81d9-ff6296b41947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:26.390148 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 4 01:00:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:00:26.676312 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a095a-edb4-4dcf-846d-4740cf39baa2 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:00:26.677898 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-759a095a-edb4-4dcf-846d-4740cf39baa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:26.779423 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a095a-edb4-4dcf-846d-4740cf39baa2 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:00:26.828393 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:26.852600 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea045b6-f227-4f2c-addd-5fc2091d3e07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:00:26.853210 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea045b6-f227-4f2c-addd-5fc2091d3e07 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:26.853493 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea045b6-f227-4f2c-addd-5fc2091d3e07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:26.853770 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea045b6-f227-4f2c-addd-5fc2091d3e07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:26.856782 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3ea045b6-f227-4f2c-addd-5fc2091d3e07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:26.967145 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea045b6-f227-4f2c-addd-5fc2091d3e07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:00:26.968664 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea045b6-f227-4f2c-addd-5fc2091d3e07 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:00:26.989074 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 726/1455] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:00:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:00:28.959173 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b55ec8-0e31-4439-b58a-99f2d40fa187 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:28.959744 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b55ec8-0e31-4439-b58a-99f2d40fa187 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:28.959744 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b55ec8-0e31-4439-b58a-99f2d40fa187 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:28.961101 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 4 01:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:00:28.966731 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c142e1-512e-442f-9521-a403d91382c6 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:28.967121 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c142e1-512e-442f-9521-a403d91382c6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:28.967284 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c142e1-512e-442f-9521-a403d91382c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:29.032936 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-38c142e1-512e-442f-9521-a403d91382c6 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:00:29.034172 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-38c142e1-512e-442f-9521-a403d91382c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:29.063501 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:29.130455 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 4 01:00:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:00:29.414677 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b55ec8-0e31-4439-b58a-99f2d40fa187 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:00:29.416263 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e4b55ec8-0e31-4439-b58a-99f2d40fa187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:29.518973 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b55ec8-0e31-4439-b58a-99f2d40fa187 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:00:29.547445 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:29.592515 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a665799f-5a9d-432b-9077-ff0bc58dcd0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:00:29.593116 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665799f-5a9d-432b-9077-ff0bc58dcd0f service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:29.593393 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665799f-5a9d-432b-9077-ff0bc58dcd0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:29.593661 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665799f-5a9d-432b-9077-ff0bc58dcd0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:29.595250 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a665799f-5a9d-432b-9077-ff0bc58dcd0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:29.702264 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a665799f-5a9d-432b-9077-ff0bc58dcd0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:00:29.703694 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a665799f-5a9d-432b-9077-ff0bc58dcd0f service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:00:29.722520 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 731/1458] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:00:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:00:31.709166 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed647f43-9251-4197-ae58-7a8b20afd37f None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:31.709735 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed647f43-9251-4197-ae58-7a8b20afd37f None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:31.709850 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed647f43-9251-4197-ae58-7a8b20afd37f None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:31.711211 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 4 01:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:00:31.717358 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a1123-a915-4e19-b516-35a17ffa7b04 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:31.717739 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a1123-a915-4e19-b516-35a17ffa7b04 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:31.717914 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a1123-a915-4e19-b516-35a17ffa7b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:31.784190 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-1a2a1123-a915-4e19-b516-35a17ffa7b04 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:00:31.785537 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-1a2a1123-a915-4e19-b516-35a17ffa7b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:31.794398 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:31.884814 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 4 01:00:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:00:32.168789 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed647f43-9251-4197-ae58-7a8b20afd37f None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:00:32.170334 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ed647f43-9251-4197-ae58-7a8b20afd37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:32.275760 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed647f43-9251-4197-ae58-7a8b20afd37f None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:00:32.331424 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:32.349727 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a743aba-35ac-4e05-a8bd-210d016d5918 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:00:32.352329 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a743aba-35ac-4e05-a8bd-210d016d5918 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:32.352329 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a743aba-35ac-4e05-a8bd-210d016d5918 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:32.352329 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a743aba-35ac-4e05-a8bd-210d016d5918 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:32.352692 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4a743aba-35ac-4e05-a8bd-210d016d5918 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:32.461719 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a743aba-35ac-4e05-a8bd-210d016d5918 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:00:32.463065 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a743aba-35ac-4e05-a8bd-210d016d5918 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:00:32.481617 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 729/1461] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:00:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:00:34.447520 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383bbd4-853c-4594-9e43-801cd9df8f50 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:34.448162 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383bbd4-853c-4594-9e43-801cd9df8f50 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:34.448162 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383bbd4-853c-4594-9e43-801cd9df8f50 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:34.449431 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 4 01:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:00:34.454728 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e6291-230a-4c5f-96ee-506988254cc9 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:34.455166 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e6291-230a-4c5f-96ee-506988254cc9 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:34.455295 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e6291-230a-4c5f-96ee-506988254cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:34.520585 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-650e6291-230a-4c5f-96ee-506988254cc9 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:00:34.521780 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-650e6291-230a-4c5f-96ee-506988254cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:34.560001 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:34.617247 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 4 01:00:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:00:34.902172 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8383bbd4-853c-4594-9e43-801cd9df8f50 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:00:34.903695 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8383bbd4-853c-4594-9e43-801cd9df8f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:35.003991 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8383bbd4-853c-4594-9e43-801cd9df8f50 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:00:35.041175 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:35.077716 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b558400-6726-4038-99d4-acd42ddf2e0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:00:35.078322 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b558400-6726-4038-99d4-acd42ddf2e0c service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:35.078601 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b558400-6726-4038-99d4-acd42ddf2e0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:35.078872 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b558400-6726-4038-99d4-acd42ddf2e0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:35.080487 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-0b558400-6726-4038-99d4-acd42ddf2e0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:35.191188 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b558400-6726-4038-99d4-acd42ddf2e0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:00:35.192588 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b558400-6726-4038-99d4-acd42ddf2e0c service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:00:35.211668 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 734/1464] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:00:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:00:37.242712 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a0c1a-ee33-43d5-ab5e-b2070d1c89b2 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:37.243269 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a0c1a-ee33-43d5-ab5e-b2070d1c89b2 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:37.243385 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a0c1a-ee33-43d5-ab5e-b2070d1c89b2 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:37.245266 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 4 01:00:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:00:37.250761 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9a0e8-3c6f-40d0-8a96-a5b32067a42d None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:37.251205 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9a0e8-3c6f-40d0-8a96-a5b32067a42d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:37.251312 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9a0e8-3c6f-40d0-8a96-a5b32067a42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:37.316840 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-1ac9a0e8-3c6f-40d0-8a96-a5b32067a42d None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:00:37.318046 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-1ac9a0e8-3c6f-40d0-8a96-a5b32067a42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:37.335795 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:37.415802 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 4 01:00:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:00:37.704053 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a0c1a-ee33-43d5-ab5e-b2070d1c89b2 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:00:37.705623 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ac0a0c1a-ee33-43d5-ab5e-b2070d1c89b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:37.808810 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a0c1a-ee33-43d5-ab5e-b2070d1c89b2 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:00:37.876474 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:37.882874 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d2dc54-490c-4c92-9dbf-106243b03778 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:00:37.883459 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d2dc54-490c-4c92-9dbf-106243b03778 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:37.883740 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d2dc54-490c-4c92-9dbf-106243b03778 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:37.884013 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d2dc54-490c-4c92-9dbf-106243b03778 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:37.886189 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-59d2dc54-490c-4c92-9dbf-106243b03778 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:37.996665 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d2dc54-490c-4c92-9dbf-106243b03778 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:00:37.998096 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d2dc54-490c-4c92-9dbf-106243b03778 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:00:38.016222 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 732/1467] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:00:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:00:39.975919 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dca745-cdcf-4a50-a478-70a5d4368e20 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:39.976508 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dca745-cdcf-4a50-a478-70a5d4368e20 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:39.976508 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dca745-cdcf-4a50-a478-70a5d4368e20 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:39.977829 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 4 01:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:00:39.983173 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48913d7b-d4eb-4f9d-8607-65e9d1b0e3b7 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:39.983610 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48913d7b-d4eb-4f9d-8607-65e9d1b0e3b7 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:39.983730 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48913d7b-d4eb-4f9d-8607-65e9d1b0e3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:40.049280 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-48913d7b-d4eb-4f9d-8607-65e9d1b0e3b7 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:00:40.050480 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-48913d7b-d4eb-4f9d-8607-65e9d1b0e3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:40.099301 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:40.147588 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 4 01:00:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:00:40.434517 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dca745-cdcf-4a50-a478-70a5d4368e20 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:00:40.436049 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-13dca745-cdcf-4a50-a478-70a5d4368e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:40.537602 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dca745-cdcf-4a50-a478-70a5d4368e20 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:00:40.585833 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:40.612915 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46756e9-3f9a-4a0e-a4d2-ef9e7dc0010c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:00:40.613540 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46756e9-3f9a-4a0e-a4d2-ef9e7dc0010c service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:00:40.613795 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46756e9-3f9a-4a0e-a4d2-ef9e7dc0010c service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:00:40.614071 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46756e9-3f9a-4a0e-a4d2-ef9e7dc0010c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:00:40.615663 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d46756e9-3f9a-4a0e-a4d2-ef9e7dc0010c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:00:40.723159 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46756e9-3f9a-4a0e-a4d2-ef9e7dc0010c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:00:40.724797 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46756e9-3f9a-4a0e-a4d2-ef9e7dc0010c service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:00:40.743740 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 737/1470] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:00:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:00:50.148286 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=22 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:00:50.741467 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=66 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:01:00.400287 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0002eda-b9f0-4c11-ad43-c0d078edde91 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:01:00.400913 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0002eda-b9f0-4c11-ad43-c0d078edde91 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:01:00.400913 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0002eda-b9f0-4c11-ad43-c0d078edde91 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:01:00.402205 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 734/1471] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 01:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:01:00.407309 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5562a5-5922-4fe0-a0d0-0c256f06ac6e None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:01:00.407754 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5562a5-5922-4fe0-a0d0-0c256f06ac6e None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:01:00.407867 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5562a5-5922-4fe0-a0d0-0c256f06ac6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:01:00.472596 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-de5562a5-5922-4fe0-a0d0-0c256f06ac6e None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:01:00.473800 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-de5562a5-5922-4fe0-a0d0-0c256f06ac6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:01:00.569897 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 738/1472] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 01:01:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:01:00.865781 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f111b6f-493f-4c3f-bfaf-4ad8a38388d4 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:01:00.866065 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f111b6f-493f-4c3f-bfaf-4ad8a38388d4 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:01:00.866339 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f111b6f-493f-4c3f-bfaf-4ad8a38388d4 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:01:00.867652 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 735/1473] 10.222.0.27 () {54 vars in 961 bytes} [Sun Sep 4 01:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:01:00.872692 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed8194-5286-4bee-9dc7-8fa200df08b0 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:01:00.872989 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed8194-5286-4bee-9dc7-8fa200df08b0 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:01:00.873264 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed8194-5286-4bee-9dc7-8fa200df08b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:01:00.950450 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-b7ed8194-5286-4bee-9dc7-8fa200df08b0 None None] MFA Rules not processed for user `1179503bdab547c99175a27b7ce27b9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:01:00.951652 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b7ed8194-5286-4bee-9dc7-8fa200df08b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:01:00.971977 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:01:01.026636 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 739/1474] 10.222.0.27 () {58 vars in 1058 bytes} [Sun Sep 4 01:01:00 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:01:01.034306 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f111b6f-493f-4c3f-bfaf-4ad8a38388d4 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:01:01.035816 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4f111b6f-493f-4c3f-bfaf-4ad8a38388d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:01:01.142115 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f111b6f-493f-4c3f-bfaf-4ad8a38388d4 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:01:01.216077 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f4a389-cb25-4c5d-91ec-dea85ebbafac service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1179503bdab547c99175a27b7ce27b9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:01:01.216708 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4a389-cb25-4c5d-91ec-dea85ebbafac service glance] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:01:01.216990 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4a389-cb25-4c5d-91ec-dea85ebbafac service glance] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:01:01.217264 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4a389-cb25-4c5d-91ec-dea85ebbafac service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:01:01.218845 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-73f4a389-cb25-4c5d-91ec-dea85ebbafac service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:01:01.274008 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:01:01.330279 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f4a389-cb25-4c5d-91ec-dea85ebbafac service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:01:01.331717 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f4a389-cb25-4c5d-91ec-dea85ebbafac service glance] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:01:01.350930 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 736/1475] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:01:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:01:03.340480 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426be241-6ada-4ee7-a082-cff4139af9bc None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:01:03.340992 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426be241-6ada-4ee7-a082-cff4139af9bc None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:01:03.341122 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426be241-6ada-4ee7-a082-cff4139af9bc None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:01:03.342372 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 740/1476] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 01:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:01:03.347627 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308c5e8-6224-4a16-9fdb-7677ef51ee4a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:01:03.348030 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308c5e8-6224-4a16-9fdb-7677ef51ee4a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:01:03.348235 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308c5e8-6224-4a16-9fdb-7677ef51ee4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:01:03.417047 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-8308c5e8-6224-4a16-9fdb-7677ef51ee4a None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:01:03.418252 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8308c5e8-6224-4a16-9fdb-7677ef51ee4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:01:03.516464 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 737/1477] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 01:01:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:01:03.797817 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426be241-6ada-4ee7-a082-cff4139af9bc None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:01:03.799297 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-426be241-6ada-4ee7-a082-cff4139af9bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:01:03.925968 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426be241-6ada-4ee7-a082-cff4139af9bc None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:01:03.975364 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:01:04.020995 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e913e07-ad7c-49a5-83d8-dcbf72f801fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:01:04.021596 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e913e07-ad7c-49a5-83d8-dcbf72f801fb admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:01:04.021869 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e913e07-ad7c-49a5-83d8-dcbf72f801fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:01:04.022136 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e913e07-ad7c-49a5-83d8-dcbf72f801fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:01:04.023026 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e913e07-ad7c-49a5-83d8-dcbf72f801fb admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:01:04.024835 dv-cinder-master-3-smb devstack@keystone.service[81630]: WARNING py.warnings [None req-7e913e07-ad7c-49a5-83d8-dcbf72f801fb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 04 01:01:04.024835 dv-cinder-master-3-smb devstack@keystone.service[81630]: warnings.warn(msg) Sep 04 01:01:04.024835 dv-cinder-master-3-smb devstack@keystone.service[81630]: Sep 04 01:01:04.026117 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e913e07-ad7c-49a5-83d8-dcbf72f801fb admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:01:04.044486 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 741/1478] 10.222.0.27 () {56 vars in 1108 bytes} [Sun Sep 4 01:01:03 2022] GET /identity/v3/projects => generated 2553 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:01:05.976381 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b2a46-6d19-47bd-b3ca-aa2f278a57ae None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:01:05.976956 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b2a46-6d19-47bd-b3ca-aa2f278a57ae None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:01:05.976956 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b2a46-6d19-47bd-b3ca-aa2f278a57ae None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:01:05.978879 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 738/1479] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 01:01:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:01:05.984107 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02781b9-2af8-4a2c-b4ae-0bea2d0209cb None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:01:05.984527 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02781b9-2af8-4a2c-b4ae-0bea2d0209cb None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:01:05.984713 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02781b9-2af8-4a2c-b4ae-0bea2d0209cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:01:06.050526 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-d02781b9-2af8-4a2c-b4ae-0bea2d0209cb None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:01:06.051713 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d02781b9-2af8-4a2c-b4ae-0bea2d0209cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:01:06.269412 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 742/1480] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 01:01:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:01:06.561995 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183b2a46-6d19-47bd-b3ca-aa2f278a57ae None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:01:06.563559 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-183b2a46-6d19-47bd-b3ca-aa2f278a57ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:01:06.573686 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:01:06.709712 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183b2a46-6d19-47bd-b3ca-aa2f278a57ae None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:01:06.789851 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:01:06.822236 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f67e5-53cc-49f7-8d4e-8e1f03cd2ea0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:01:06.822837 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f67e5-53cc-49f7-8d4e-8e1f03cd2ea0 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:01:06.823112 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f67e5-53cc-49f7-8d4e-8e1f03cd2ea0 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:01:06.823379 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f67e5-53cc-49f7-8d4e-8e1f03cd2ea0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:01:06.825752 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b18f67e5-53cc-49f7-8d4e-8e1f03cd2ea0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:01:06.916600 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f67e5-53cc-49f7-8d4e-8e1f03cd2ea0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:01:06.918247 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f67e5-53cc-49f7-8d4e-8e1f03cd2ea0 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:01:06.936329 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 739/1481] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:01:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:01:09.010970 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d38f06-34a2-4539-a9e9-1c231cc96a2d None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:01:09.011550 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d38f06-34a2-4539-a9e9-1c231cc96a2d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:01:09.011550 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d38f06-34a2-4539-a9e9-1c231cc96a2d None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:01:09.012858 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 743/1482] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 01:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:01:09.017919 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d8402-ac86-46a7-8d21-e46f80914dd2 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:01:09.018320 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d8402-ac86-46a7-8d21-e46f80914dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:01:09.018483 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d8402-ac86-46a7-8d21-e46f80914dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:01:09.085433 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-898d8402-ac86-46a7-8d21-e46f80914dd2 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:01:09.086634 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-898d8402-ac86-46a7-8d21-e46f80914dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:01:09.134810 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:01:09.186471 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 740/1483] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 01:01:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:01:09.476786 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d38f06-34a2-4539-a9e9-1c231cc96a2d None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:01:09.478397 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c3d38f06-34a2-4539-a9e9-1c231cc96a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:01:09.501081 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:01:09.581362 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d38f06-34a2-4539-a9e9-1c231cc96a2d None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:01:09.655980 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d120835b-b47c-44e8-a8f0-f6c04e510527 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:01:09.656620 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120835b-b47c-44e8-a8f0-f6c04e510527 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:01:09.656904 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120835b-b47c-44e8-a8f0-f6c04e510527 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:01:09.657175 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120835b-b47c-44e8-a8f0-f6c04e510527 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:01:09.658773 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d120835b-b47c-44e8-a8f0-f6c04e510527 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:01:09.732731 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:01:09.768200 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d120835b-b47c-44e8-a8f0-f6c04e510527 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:01:09.769739 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d120835b-b47c-44e8-a8f0-f6c04e510527 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:01:09.788155 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 744/1484] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:01:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:01:11.786543 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3409a4-fa5f-4305-8110-f0b507509c2e None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:01:11.787113 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3409a4-fa5f-4305-8110-f0b507509c2e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:01:11.787113 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3409a4-fa5f-4305-8110-f0b507509c2e None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:01:11.788443 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 741/1485] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 01:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:01:11.794372 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93713ae9-a4d8-4a6c-8dd1-6ee1577f2b0a None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:01:11.794813 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93713ae9-a4d8-4a6c-8dd1-6ee1577f2b0a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:01:11.794922 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93713ae9-a4d8-4a6c-8dd1-6ee1577f2b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:01:11.859077 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-93713ae9-a4d8-4a6c-8dd1-6ee1577f2b0a None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:01:11.860283 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-93713ae9-a4d8-4a6c-8dd1-6ee1577f2b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:01:11.955544 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 745/1486] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 01:01:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:01:12.242019 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3409a4-fa5f-4305-8110-f0b507509c2e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:01:12.243569 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bb3409a4-fa5f-4305-8110-f0b507509c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:01:12.348274 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3409a4-fa5f-4305-8110-f0b507509c2e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:01:12.419323 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:01:12.421120 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac984d0-0011-492c-90dc-b5364a19138c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:01:12.421720 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac984d0-0011-492c-90dc-b5364a19138c service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:01:12.422005 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac984d0-0011-492c-90dc-b5364a19138c service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:01:12.422282 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac984d0-0011-492c-90dc-b5364a19138c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:01:12.423870 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9ac984d0-0011-492c-90dc-b5364a19138c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:01:12.536302 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac984d0-0011-492c-90dc-b5364a19138c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:01:12.537890 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac984d0-0011-492c-90dc-b5364a19138c service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:01:12.555808 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 742/1487] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:01:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:01:14.546515 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc681c5-100d-497f-bc6a-c58c6a8814e6 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:01:14.547035 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc681c5-100d-497f-bc6a-c58c6a8814e6 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:01:14.547035 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc681c5-100d-497f-bc6a-c58c6a8814e6 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:01:14.548372 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 746/1488] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 01:01:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:01:14.553540 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bef68b-47d4-4df5-b022-01bc80c16eae None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:01:14.553964 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bef68b-47d4-4df5-b022-01bc80c16eae None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:01:14.554130 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bef68b-47d4-4df5-b022-01bc80c16eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:01:14.619734 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f2bef68b-47d4-4df5-b022-01bc80c16eae None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:01:14.620968 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f2bef68b-47d4-4df5-b022-01bc80c16eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:01:14.673758 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:01:14.718848 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 743/1489] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 01:01:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:01:15.009245 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc681c5-100d-497f-bc6a-c58c6a8814e6 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:01:15.010881 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-cdc681c5-100d-497f-bc6a-c58c6a8814e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:01:15.046673 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:01:15.113160 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc681c5-100d-497f-bc6a-c58c6a8814e6 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:01:15.184848 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980021e7-7fdd-4e63-8cdf-5435427519ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:01:15.185447 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980021e7-7fdd-4e63-8cdf-5435427519ad service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:01:15.185721 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980021e7-7fdd-4e63-8cdf-5435427519ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:01:15.185990 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980021e7-7fdd-4e63-8cdf-5435427519ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:01:15.187583 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-980021e7-7fdd-4e63-8cdf-5435427519ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:01:15.265976 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:01:15.294617 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980021e7-7fdd-4e63-8cdf-5435427519ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:01:15.296058 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980021e7-7fdd-4e63-8cdf-5435427519ad service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:01:15.314284 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 747/1490] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:01:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:01:17.718174 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7027b-7ef8-40e3-ac9a-67c43d4f27da None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:01:17.718759 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7027b-7ef8-40e3-ac9a-67c43d4f27da None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:01:17.718759 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7027b-7ef8-40e3-ac9a-67c43d4f27da None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:01:17.720069 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 744/1491] 10.222.0.27 () {54 vars in 927 bytes} [Sun Sep 4 01:01:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:01:17.725325 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba98459-0a94-41fb-8b04-be8fae2684d7 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:01:17.725732 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba98459-0a94-41fb-8b04-be8fae2684d7 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:01:17.725885 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba98459-0a94-41fb-8b04-be8fae2684d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:01:17.793803 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-8ba98459-0a94-41fb-8b04-be8fae2684d7 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:01:17.795028 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8ba98459-0a94-41fb-8b04-be8fae2684d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:01:17.891862 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 748/1492] 10.222.0.27 () {58 vars in 1024 bytes} [Sun Sep 4 01:01:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:01:18.184621 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7027b-7ef8-40e3-ac9a-67c43d4f27da None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:01:18.186169 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-10e7027b-7ef8-40e3-ac9a-67c43d4f27da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:01:18.291655 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7027b-7ef8-40e3-ac9a-67c43d4f27da None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:01:18.367120 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae42d6f-0411-4af9-a69d-f94039aff234 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:01:18.367817 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae42d6f-0411-4af9-a69d-f94039aff234 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:01:18.368616 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae42d6f-0411-4af9-a69d-f94039aff234 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:01:18.368887 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae42d6f-0411-4af9-a69d-f94039aff234 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:01:18.370466 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-cae42d6f-0411-4af9-a69d-f94039aff234 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:01:18.379590 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:01:18.478813 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae42d6f-0411-4af9-a69d-f94039aff234 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:01:18.480147 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae42d6f-0411-4af9-a69d-f94039aff234 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:01:18.498374 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 745/1493] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:01:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:01:27.890043 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=82 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:01:28.496105 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=50 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:02:39.854901 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0240e-1ea0-498b-9f2b-01a124449a9a None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:02:39.855562 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0240e-1ea0-498b-9f2b-01a124449a9a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:02:39.855562 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0240e-1ea0-498b-9f2b-01a124449a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:02:39.931339 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-71c0240e-1ea0-498b-9f2b-01a124449a9a None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:02:39.932563 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-71c0240e-1ea0-498b-9f2b-01a124449a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:02:40.025921 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 749/1494] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:02:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:02:40.033878 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b3796-574f-4172-9319-f1eb248003f3 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:02:40.034388 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b3796-574f-4172-9319-f1eb248003f3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:02:40.034503 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b3796-574f-4172-9319-f1eb248003f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:02:40.110149 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-884b3796-574f-4172-9319-f1eb248003f3 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:02:40.111288 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-884b3796-574f-4172-9319-f1eb248003f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:02:40.207824 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 746/1495] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:02:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:02:40.212430 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0240e-1ea0-498b-9f2b-01a124449a9a None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:02:40.213662 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-71c0240e-1ea0-498b-9f2b-01a124449a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:02:40.225647 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0240e-1ea0-498b-9f2b-01a124449a9a None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:02:40.285288 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:02:40.318878 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c9bbbe-3193-4ca2-be29-a548fc58708c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:02:40.319480 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9bbbe-3193-4ca2-be29-a548fc58708c admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:02:40.319756 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9bbbe-3193-4ca2-be29-a548fc58708c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:02:40.320025 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9bbbe-3193-4ca2-be29-a548fc58708c admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:02:40.320684 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c9bbbe-3193-4ca2-be29-a548fc58708c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:02:40.320983 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c9bbbe-3193-4ca2-be29-a548fc58708c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:02:40.321988 dv-cinder-master-3-smb devstack@keystone.service[81630]: WARNING py.warnings [None req-77c9bbbe-3193-4ca2-be29-a548fc58708c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 04 01:02:40.321988 dv-cinder-master-3-smb devstack@keystone.service[81630]: warnings.warn(msg) Sep 04 01:02:40.321988 dv-cinder-master-3-smb devstack@keystone.service[81630]: Sep 04 01:02:40.322892 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c9bbbe-3193-4ca2-be29-a548fc58708c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:02:40.333810 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 750/1496] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:02:40.342127 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b3796-574f-4172-9319-f1eb248003f3 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:02:40.354251 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b3796-574f-4172-9319-f1eb248003f3 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:02:40.416642 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:02:40.449719 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdbadb1-6413-4bb4-8035-9dfd52fd3fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:02:40.450321 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbadb1-6413-4bb4-8035-9dfd52fd3fe3 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:02:40.450600 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbadb1-6413-4bb4-8035-9dfd52fd3fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:02:40.450872 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbadb1-6413-4bb4-8035-9dfd52fd3fe3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:02:40.451504 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdbadb1-6413-4bb4-8035-9dfd52fd3fe3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:02:40.453231 dv-cinder-master-3-smb devstack@keystone.service[81629]: WARNING py.warnings [None req-afdbadb1-6413-4bb4-8035-9dfd52fd3fe3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 04 01:02:40.453231 dv-cinder-master-3-smb devstack@keystone.service[81629]: warnings.warn(msg) Sep 04 01:02:40.453231 dv-cinder-master-3-smb devstack@keystone.service[81629]: Sep 04 01:02:40.454423 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdbadb1-6413-4bb4-8035-9dfd52fd3fe3 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:02:40.639088 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 747/1497] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:02:40 2022] POST /identity/v3/projects => generated 379 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:02:40.643285 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c9bbbe-3193-4ca2-be29-a548fc58708c admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:02:40.644856 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-77c9bbbe-3193-4ca2-be29-a548fc58708c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:02:40.679161 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c9bbbe-3193-4ca2-be29-a548fc58708c admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:02:40.774081 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12de07cf-c901-456f-be36-3c3d1ef6f8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:02:40.774674 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de07cf-c901-456f-be36-3c3d1ef6f8ca admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:02:40.774951 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de07cf-c901-456f-be36-3c3d1ef6f8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:02:40.775219 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de07cf-c901-456f-be36-3c3d1ef6f8ca admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:02:40.775834 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12de07cf-c901-456f-be36-3c3d1ef6f8ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:02:40.776880 dv-cinder-master-3-smb devstack@keystone.service[81630]: WARNING py.warnings [None req-12de07cf-c901-456f-be36-3c3d1ef6f8ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 04 01:02:40.776880 dv-cinder-master-3-smb devstack@keystone.service[81630]: warnings.warn(msg) Sep 04 01:02:40.776880 dv-cinder-master-3-smb devstack@keystone.service[81630]: Sep 04 01:02:40.778095 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12de07cf-c901-456f-be36-3c3d1ef6f8ca admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:02:40.888343 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 751/1498] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:02:40 2022] POST /identity/v3/users => generated 290 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:02:40.892577 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdbadb1-6413-4bb4-8035-9dfd52fd3fe3 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:02:40.894121 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-afdbadb1-6413-4bb4-8035-9dfd52fd3fe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:02:40.956628 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdbadb1-6413-4bb4-8035-9dfd52fd3fe3 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:02:41.041008 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:02:41.051033 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf029f17-153a-410a-b726-4886d87f6397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:02:41.051643 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf029f17-153a-410a-b726-4886d87f6397 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:02:41.051903 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf029f17-153a-410a-b726-4886d87f6397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:02:41.052183 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf029f17-153a-410a-b726-4886d87f6397 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:02:41.053236 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf029f17-153a-410a-b726-4886d87f6397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:02:41.054243 dv-cinder-master-3-smb devstack@keystone.service[81629]: WARNING py.warnings [None req-bf029f17-153a-410a-b726-4886d87f6397 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 04 01:02:41.054243 dv-cinder-master-3-smb devstack@keystone.service[81629]: warnings.warn(msg) Sep 04 01:02:41.054243 dv-cinder-master-3-smb devstack@keystone.service[81629]: Sep 04 01:02:41.055130 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf029f17-153a-410a-b726-4886d87f6397 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:02:41.067251 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 748/1499] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 01:02:41.071963 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12de07cf-c901-456f-be36-3c3d1ef6f8ca admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:02:41.073596 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-12de07cf-c901-456f-be36-3c3d1ef6f8ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:02:41.085737 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12de07cf-c901-456f-be36-3c3d1ef6f8ca admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:02:41.110501 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:02:41.180540 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3979778d-2201-4913-9f60-4d204f3af987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:02:41.181143 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979778d-2201-4913-9f60-4d204f3af987 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:02:41.181423 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979778d-2201-4913-9f60-4d204f3af987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:02:41.181697 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979778d-2201-4913-9f60-4d204f3af987 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:02:41.182334 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3979778d-2201-4913-9f60-4d204f3af987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:02:41.183375 dv-cinder-master-3-smb devstack@keystone.service[81630]: WARNING py.warnings [None req-3979778d-2201-4913-9f60-4d204f3af987 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 04 01:02:41.183375 dv-cinder-master-3-smb devstack@keystone.service[81630]: warnings.warn(msg) Sep 04 01:02:41.183375 dv-cinder-master-3-smb devstack@keystone.service[81630]: Sep 04 01:02:41.184254 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3979778d-2201-4913-9f60-4d204f3af987 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:02:41.197111 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 752/1500] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:02:41.201282 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf029f17-153a-410a-b726-4886d87f6397 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:02:41.202792 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bf029f17-153a-410a-b726-4886d87f6397 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:02:41.236063 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf029f17-153a-410a-b726-4886d87f6397 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:02:41.331046 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61ce45-489f-4e43-9ecb-ff105446aa9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:02:41.331708 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61ce45-489f-4e43-9ecb-ff105446aa9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:02:41.331990 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61ce45-489f-4e43-9ecb-ff105446aa9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:02:41.332264 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61ce45-489f-4e43-9ecb-ff105446aa9e admin admin] PATH_INFO: `/v3/projects/522c5ca7feea4e958d4c56ea0052a249/users/61490dc608e7454ea6d738ad377e8c47/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:02:41.354603 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a61ce45-489f-4e43-9ecb-ff105446aa9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=522c5ca7feea4e958d4c56ea0052a249, user_id=61490dc608e7454ea6d738ad377e8c47, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:02:41.355611 dv-cinder-master-3-smb devstack@keystone.service[81629]: WARNING py.warnings [None req-9a61ce45-489f-4e43-9ecb-ff105446aa9e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 04 01:02:41.355611 dv-cinder-master-3-smb devstack@keystone.service[81629]: warnings.warn(msg) Sep 04 01:02:41.355611 dv-cinder-master-3-smb devstack@keystone.service[81629]: Sep 04 01:02:41.357578 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a61ce45-489f-4e43-9ecb-ff105446aa9e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:02:41.441909 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 749/1501] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:02:41 2022] PUT /identity/v3/projects/522c5ca7feea4e958d4c56ea0052a249/users/61490dc608e7454ea6d738ad377e8c47/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:02:41.454921 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8e1de-3044-4a50-8c65-4d9ce1cdfd88 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:02:41.455209 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8e1de-3044-4a50-8c65-4d9ce1cdfd88 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:02:41.455487 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8e1de-3044-4a50-8c65-4d9ce1cdfd88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:02:41.512614 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-47e8e1de-3044-4a50-8c65-4d9ce1cdfd88 None None] MFA Rules not processed for user `61490dc608e7454ea6d738ad377e8c47`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:02:41.517109 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-47e8e1de-3044-4a50-8c65-4d9ce1cdfd88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:02:41.591540 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 753/1502] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:02:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:02:41.605080 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06bff45-8d1b-4288-a7bf-005bb3948cef None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:02:41.605371 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06bff45-8d1b-4288-a7bf-005bb3948cef None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:02:41.605649 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06bff45-8d1b-4288-a7bf-005bb3948cef None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:02:41.606982 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 750/1503] 10.222.0.27 () {54 vars in 964 bytes} [Sun Sep 4 01:02:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:02:41.611790 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95276405-6574-4dfd-ad41-7f715d94ea54 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:02:41.612086 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95276405-6574-4dfd-ad41-7f715d94ea54 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:02:41.612360 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95276405-6574-4dfd-ad41-7f715d94ea54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:02:41.644177 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:02:41.689472 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-95276405-6574-4dfd-ad41-7f715d94ea54 None None] MFA Rules not processed for user `6297dbc521e94330ae9294b942144181`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:02:41.690672 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-95276405-6574-4dfd-ad41-7f715d94ea54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:02:41.764852 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 754/1504] 10.222.0.27 () {58 vars in 1061 bytes} [Sun Sep 4 01:02:41 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:02:41.773485 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06bff45-8d1b-4288-a7bf-005bb3948cef None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:02:41.774999 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f06bff45-8d1b-4288-a7bf-005bb3948cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:02:41.825266 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:02:41.878567 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06bff45-8d1b-4288-a7bf-005bb3948cef None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:02:41.953468 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5957d2-196c-4a88-bb7a-40f02877d313 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:02:41.954075 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5957d2-196c-4a88-bb7a-40f02877d313 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:02:41.954358 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5957d2-196c-4a88-bb7a-40f02877d313 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:02:41.954634 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5957d2-196c-4a88-bb7a-40f02877d313 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:02:41.956235 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ee5957d2-196c-4a88-bb7a-40f02877d313 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:02:42.054720 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5957d2-196c-4a88-bb7a-40f02877d313 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:02:42.056110 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5957d2-196c-4a88-bb7a-40f02877d313 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:02:42.059695 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:02:42.074346 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 751/1505] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:02:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:02:42.121230 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8050e3b-f048-4900-b9f4-54747fb276f7 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:02:42.121668 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8050e3b-f048-4900-b9f4-54747fb276f7 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:02:42.121791 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8050e3b-f048-4900-b9f4-54747fb276f7 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:02:42.123048 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 755/1506] 10.222.0.27 () {52 vars in 830 bytes} [Sun Sep 4 01:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:02:42.126787 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5957d2-196c-4a88-bb7a-40f02877d313 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:02:42.128065 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ee5957d2-196c-4a88-bb7a-40f02877d313 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:02:42.231987 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5957d2-196c-4a88-bb7a-40f02877d313 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:02:42.326816 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57215130-d7c5-4851-8da3-d9112dc2ccda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:02:42.327366 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57215130-d7c5-4851-8da3-d9112dc2ccda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:02:42.327644 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57215130-d7c5-4851-8da3-d9112dc2ccda admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:02:42.327916 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57215130-d7c5-4851-8da3-d9112dc2ccda admin admin] PATH_INFO: `/v3/users/61490dc608e7454ea6d738ad377e8c47` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:02:42.340875 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57215130-d7c5-4851-8da3-d9112dc2ccda admin admin] RBAC: Authorizing `identity:delete_user(user_id=61490dc608e7454ea6d738ad377e8c47)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:02:42.341889 dv-cinder-master-3-smb devstack@keystone.service[81629]: WARNING py.warnings [None req-57215130-d7c5-4851-8da3-d9112dc2ccda admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 04 01:02:42.341889 dv-cinder-master-3-smb devstack@keystone.service[81629]: warnings.warn(msg) Sep 04 01:02:42.341889 dv-cinder-master-3-smb devstack@keystone.service[81629]: Sep 04 01:02:42.343109 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57215130-d7c5-4851-8da3-d9112dc2ccda admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:02:42.353290 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:02:42.537306 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-57215130-d7c5-4851-8da3-d9112dc2ccda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61490dc608e7454ea6d738ad377e8c47'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:02:42.537965 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-57215130-d7c5-4851-8da3-d9112dc2ccda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61490dc608e7454ea6d738ad377e8c47'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:02:42.552085 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-57215130-d7c5-4851-8da3-d9112dc2ccda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61490dc608e7454ea6d738ad377e8c47'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:02:42.657413 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-57215130-d7c5-4851-8da3-d9112dc2ccda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61490dc608e7454ea6d738ad377e8c47'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:02:42.657890 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-57215130-d7c5-4851-8da3-d9112dc2ccda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61490dc608e7454ea6d738ad377e8c47'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:02:42.687826 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 752/1507] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:02:42 2022] DELETE /identity/v3/users/61490dc608e7454ea6d738ad377e8c47 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:02:42.692254 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa32307-a18b-4162-9612-d0f09b4f23cc None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:02:42.692585 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa32307-a18b-4162-9612-d0f09b4f23cc None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:02:42.692870 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa32307-a18b-4162-9612-d0f09b4f23cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:02:42.749160 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-9aa32307-a18b-4162-9612-d0f09b4f23cc None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:02:42.750364 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-9aa32307-a18b-4162-9612-d0f09b4f23cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:02:42.823643 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:02:42.836281 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 756/1508] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:02:42.845960 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57215130-d7c5-4851-8da3-d9112dc2ccda admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:02:42.847177 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-57215130-d7c5-4851-8da3-d9112dc2ccda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:02:42.950615 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57215130-d7c5-4851-8da3-d9112dc2ccda admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:02:43.004869 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:02:43.024231 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8467a73e-7d47-4485-928c-b34fc4d71d41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:02:43.024869 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8467a73e-7d47-4485-928c-b34fc4d71d41 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:02:43.025150 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8467a73e-7d47-4485-928c-b34fc4d71d41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:02:43.025440 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8467a73e-7d47-4485-928c-b34fc4d71d41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:02:43.027026 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8467a73e-7d47-4485-928c-b34fc4d71d41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:02:43.135174 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8467a73e-7d47-4485-928c-b34fc4d71d41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:02:43.136626 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8467a73e-7d47-4485-928c-b34fc4d71d41 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:02:43.155622 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 753/1509] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:02:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:02:43.620755 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa32307-a18b-4162-9612-d0f09b4f23cc None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:02:43.622101 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-9aa32307-a18b-4162-9612-d0f09b4f23cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:02:43.633795 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa32307-a18b-4162-9612-d0f09b4f23cc None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:02:43.739585 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3b5d5-a855-437e-99b9-233b866c0779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:02:43.740280 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3b5d5-a855-437e-99b9-233b866c0779 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:02:43.740660 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3b5d5-a855-437e-99b9-233b866c0779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:02:43.740937 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3b5d5-a855-437e-99b9-233b866c0779 admin admin] PATH_INFO: `/v3/projects/522c5ca7feea4e958d4c56ea0052a249` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:02:43.750777 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f3b5d5-a855-437e-99b9-233b866c0779 admin admin] RBAC: Authorizing `identity:delete_project(project_id=522c5ca7feea4e958d4c56ea0052a249)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:02:43.751813 dv-cinder-master-3-smb devstack@keystone.service[81630]: WARNING py.warnings [None req-b6f3b5d5-a855-437e-99b9-233b866c0779 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 04 01:02:43.751813 dv-cinder-master-3-smb devstack@keystone.service[81630]: warnings.warn(msg) Sep 04 01:02:43.751813 dv-cinder-master-3-smb devstack@keystone.service[81630]: Sep 04 01:02:43.753122 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f3b5d5-a855-437e-99b9-233b866c0779 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:02:43.855468 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b6f3b5d5-a855-437e-99b9-233b866c0779 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '522c5ca7feea4e958d4c56ea0052a249', 'request_id': 'req-b6f3b5d5-a855-437e-99b9-233b866c0779'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:02:43.931080 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b6f3b5d5-a855-437e-99b9-233b866c0779 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '522c5ca7feea4e958d4c56ea0052a249', 'request_id': 'req-b6f3b5d5-a855-437e-99b9-233b866c0779'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:02:43.937437 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b6f3b5d5-a855-437e-99b9-233b866c0779 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '522c5ca7feea4e958d4c56ea0052a249', 'request_id': 'req-b6f3b5d5-a855-437e-99b9-233b866c0779'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:02:43.967739 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b6f3b5d5-a855-437e-99b9-233b866c0779 admin admin] The token cache is being invalidate because project 522c5ca7feea4e958d4c56ea0052a249 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:02:43.968029 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b6f3b5d5-a855-437e-99b9-233b866c0779 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:02:43.968475 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b6f3b5d5-a855-437e-99b9-233b866c0779 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:02:43.969888 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 757/1510] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:02:43 2022] DELETE /identity/v3/projects/522c5ca7feea4e958d4c56ea0052a249 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:02:53.153369 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=62 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:02:53.965801 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:04:20.541929 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55e684-b6cb-4e05-b6eb-c8a153106ea8 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:04:20.542538 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55e684-b6cb-4e05-b6eb-c8a153106ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:04:20.542538 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55e684-b6cb-4e05-b6eb-c8a153106ea8 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:04:20.543873 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 4 01:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:04:20.548716 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84639e79-b5a0-49ed-9db7-8f7c9f9e7399 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:04:20.549150 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84639e79-b5a0-49ed-9db7-8f7c9f9e7399 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:04:20.549255 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84639e79-b5a0-49ed-9db7-8f7c9f9e7399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:04:20.628429 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-84639e79-b5a0-49ed-9db7-8f7c9f9e7399 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:04:20.629781 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-84639e79-b5a0-49ed-9db7-8f7c9f9e7399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:04:20.732296 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 4 01:04:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:04:20.890400 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af55e684-b6cb-4e05-b6eb-c8a153106ea8 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:04:20.892534 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-af55e684-b6cb-4e05-b6eb-c8a153106ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:04:21.012620 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af55e684-b6cb-4e05-b6eb-c8a153106ea8 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:04:21.018288 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d551079-648f-4081-a012-098813161e2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:04:21.018884 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d551079-648f-4081-a012-098813161e2e service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:04:21.019162 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d551079-648f-4081-a012-098813161e2e service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:04:21.019434 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d551079-648f-4081-a012-098813161e2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:04:21.029764 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d551079-648f-4081-a012-098813161e2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:04:21.031373 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d551079-648f-4081-a012-098813161e2e service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:04:21.041421 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 755/1513] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:04:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:04:21.874628 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e08385-1e84-4d06-bfcd-155a9e11052b None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:04:21.875202 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e08385-1e84-4d06-bfcd-155a9e11052b None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:04:21.875202 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e08385-1e84-4d06-bfcd-155a9e11052b None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:04:21.876551 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 759/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 4 01:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:04:21.882049 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083772d8-bf43-4128-98d7-7d7383d04f9a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:04:21.882467 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083772d8-bf43-4128-98d7-7d7383d04f9a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:04:21.882629 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083772d8-bf43-4128-98d7-7d7383d04f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:04:21.903063 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-083772d8-bf43-4128-98d7-7d7383d04f9a None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:04:21.904749 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-083772d8-bf43-4128-98d7-7d7383d04f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:04:21.912986 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 756/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 4 01:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:04:22.065351 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e08385-1e84-4d06-bfcd-155a9e11052b None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:04:22.079924 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e08385-1e84-4d06-bfcd-155a9e11052b None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:04:22.085952 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c3f3d-f699-427e-b12f-e4bf020bc470 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:04:22.086536 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c3f3d-f699-427e-b12f-e4bf020bc470 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:04:22.086814 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c3f3d-f699-427e-b12f-e4bf020bc470 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:04:22.087085 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c3f3d-f699-427e-b12f-e4bf020bc470 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:04:22.095462 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c3f3d-f699-427e-b12f-e4bf020bc470 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:04:22.097062 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c3f3d-f699-427e-b12f-e4bf020bc470 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:04:22.106748 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 760/1516] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:04:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:04:22.648333 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bccbc4-9b03-4b50-9883-6d8e96a9304a None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:04:22.648763 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bccbc4-9b03-4b50-9883-6d8e96a9304a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:04:22.649057 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bccbc4-9b03-4b50-9883-6d8e96a9304a None None] PATH_INFO: `/v3` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:04:22.650335 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 757/1517] 10.222.0.55 () {54 vars in 925 bytes} [Sun Sep 4 01:04:22 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:04:22.657545 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a46abd-193f-4336-94ba-58105ad88e84 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:04:22.657838 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a46abd-193f-4336-94ba-58105ad88e84 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:04:22.658112 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a46abd-193f-4336-94ba-58105ad88e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:04:22.720881 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-34a46abd-193f-4336-94ba-58105ad88e84 None None] MFA Rules not processed for user `018802a8d9c041119ad708430f885a89`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:04:22.722104 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-34a46abd-193f-4336-94ba-58105ad88e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:04:22.740003 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:04:22.795336 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 761/1518] 10.222.0.55 () {58 vars in 1015 bytes} [Sun Sep 4 01:04:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:04:31.898181 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=56 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:04:31.942886 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a31b2b-91f3-4ee2-ac31-1cd4148b2801 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:04:31.943098 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a31b2b-91f3-4ee2-ac31-1cd4148b2801 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:04:31.943371 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a31b2b-91f3-4ee2-ac31-1cd4148b2801 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:04:31.944735 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 758/1519] 10.222.0.27 () {54 vars in 961 bytes} [Sun Sep 4 01:04:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:04:31.950062 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90ca5d-7261-4490-a026-5d5582168e9c None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:04:31.950496 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90ca5d-7261-4490-a026-5d5582168e9c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:04:31.950617 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90ca5d-7261-4490-a026-5d5582168e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:04:31.969950 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-0e90ca5d-7261-4490-a026-5d5582168e9c None None] MFA Rules not processed for user `018802a8d9c041119ad708430f885a89`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:04:31.971151 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-0e90ca5d-7261-4490-a026-5d5582168e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:04:31.980480 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 762/1520] 10.222.0.27 () {58 vars in 1058 bytes} [Sun Sep 4 01:04:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:04:31.987902 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a31b2b-91f3-4ee2-ac31-1cd4148b2801 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:04:32.001710 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a31b2b-91f3-4ee2-ac31-1cd4148b2801 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:04:32.006763 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d325791-a5ad-44de-b043-2de985984f26 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '018802a8d9c041119ad708430f885a89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:04:32.007344 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d325791-a5ad-44de-b043-2de985984f26 service placement] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:04:32.007622 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d325791-a5ad-44de-b043-2de985984f26 service placement] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:04:32.007897 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d325791-a5ad-44de-b043-2de985984f26 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:04:32.015902 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d325791-a5ad-44de-b043-2de985984f26 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:04:32.017497 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d325791-a5ad-44de-b043-2de985984f26 service placement] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:04:32.027043 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 759/1521] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:04:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:04:33.340593 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e27434-cfa0-4d64-91d6-855440134296 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:04:33.341154 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e27434-cfa0-4d64-91d6-855440134296 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:04:33.341154 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e27434-cfa0-4d64-91d6-855440134296 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:04:33.342479 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 4 01:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:04:33.347264 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fc34f-bab9-409b-a2e0-b18e985bb98f None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:04:33.347666 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fc34f-bab9-409b-a2e0-b18e985bb98f None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:04:33.347841 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fc34f-bab9-409b-a2e0-b18e985bb98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:04:33.368393 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-869fc34f-bab9-409b-a2e0-b18e985bb98f None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:04:33.369609 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-869fc34f-bab9-409b-a2e0-b18e985bb98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:04:33.377341 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 4 01:04:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:04:33.528561 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e27434-cfa0-4d64-91d6-855440134296 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:04:33.543456 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e27434-cfa0-4d64-91d6-855440134296 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:04:33.548968 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce54b1bf-95d9-45c1-9211-60a40ecdd2a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:04:33.549564 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54b1bf-95d9-45c1-9211-60a40ecdd2a3 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:04:33.549849 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54b1bf-95d9-45c1-9211-60a40ecdd2a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:04:33.550146 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54b1bf-95d9-45c1-9211-60a40ecdd2a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:04:33.559028 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce54b1bf-95d9-45c1-9211-60a40ecdd2a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:04:33.560734 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce54b1bf-95d9-45c1-9211-60a40ecdd2a3 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:04:33.570315 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 764/1524] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:04:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:04:34.383483 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff463e-2d54-468a-8d46-fcbf22c3217b None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:04:34.384061 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff463e-2d54-468a-8d46-fcbf22c3217b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:04:34.384061 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff463e-2d54-468a-8d46-fcbf22c3217b None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:04:34.385397 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 4 01:04:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:04:34.390095 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d3585-3800-47a7-88b3-b2308423f2b1 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:04:34.390513 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d3585-3800-47a7-88b3-b2308423f2b1 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:04:34.390641 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d3585-3800-47a7-88b3-b2308423f2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:04:34.409931 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-cd6d3585-3800-47a7-88b3-b2308423f2b1 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:04:34.411182 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-cd6d3585-3800-47a7-88b3-b2308423f2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:04:34.418909 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 4 01:04:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:04:34.569101 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfff463e-2d54-468a-8d46-fcbf22c3217b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:04:34.583285 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfff463e-2d54-468a-8d46-fcbf22c3217b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:04:34.588739 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb794f4c-c35a-4f10-94fd-b52f1b350d62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:04:34.589333 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb794f4c-c35a-4f10-94fd-b52f1b350d62 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:04:34.589622 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb794f4c-c35a-4f10-94fd-b52f1b350d62 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:04:34.589895 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb794f4c-c35a-4f10-94fd-b52f1b350d62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:04:34.597827 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb794f4c-c35a-4f10-94fd-b52f1b350d62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:04:34.599398 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb794f4c-c35a-4f10-94fd-b52f1b350d62 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:04:34.608664 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 762/1527] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:04:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:04:38.522000 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6561a59-0da1-44c4-842b-729507ac06d1 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:04:38.522538 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6561a59-0da1-44c4-842b-729507ac06d1 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:04:38.522538 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6561a59-0da1-44c4-842b-729507ac06d1 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:04:38.523881 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 4 01:04:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:04:38.529087 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e933e-4cef-4c83-8d56-a0ffdb5f91a8 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:04:38.529539 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e933e-4cef-4c83-8d56-a0ffdb5f91a8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:04:38.529650 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e933e-4cef-4c83-8d56-a0ffdb5f91a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:04:38.550080 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-cb1e933e-4cef-4c83-8d56-a0ffdb5f91a8 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:04:38.551282 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-cb1e933e-4cef-4c83-8d56-a0ffdb5f91a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:04:38.559848 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 4 01:04:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:04:38.857205 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6561a59-0da1-44c4-842b-729507ac06d1 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:04:38.858872 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b6561a59-0da1-44c4-842b-729507ac06d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:04:38.938099 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6561a59-0da1-44c4-842b-729507ac06d1 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:04:38.941702 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6898f1f-478d-4604-96f9-78bd6270e046 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:04:38.942305 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6898f1f-478d-4604-96f9-78bd6270e046 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:04:38.943190 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6898f1f-478d-4604-96f9-78bd6270e046 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:04:38.943542 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6898f1f-478d-4604-96f9-78bd6270e046 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:04:38.953286 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6898f1f-478d-4604-96f9-78bd6270e046 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:04:38.954730 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6898f1f-478d-4604-96f9-78bd6270e046 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:04:38.964193 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 767/1530] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:04:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:04:40.841266 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823827b-35fd-4209-abee-fa07991a790f None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:04:40.841781 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823827b-35fd-4209-abee-fa07991a790f None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:04:40.841877 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823827b-35fd-4209-abee-fa07991a790f None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:04:40.843456 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 4 01:04:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:04:40.849164 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4e00b-dbcb-4d2d-8fd8-69c62e0de405 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:04:40.849628 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4e00b-dbcb-4d2d-8fd8-69c62e0de405 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:04:40.849810 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4e00b-dbcb-4d2d-8fd8-69c62e0de405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:04:40.869948 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-60b4e00b-dbcb-4d2d-8fd8-69c62e0de405 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:04:40.871172 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-60b4e00b-dbcb-4d2d-8fd8-69c62e0de405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:04:40.879272 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 4 01:04:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:04:41.172536 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823827b-35fd-4209-abee-fa07991a790f None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:04:41.174233 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1823827b-35fd-4209-abee-fa07991a790f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:04:41.187202 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823827b-35fd-4209-abee-fa07991a790f None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:04:41.190737 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591da9f7-3fef-4a63-9f07-c6aa1a575385 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:04:41.191329 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591da9f7-3fef-4a63-9f07-c6aa1a575385 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:04:41.191610 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591da9f7-3fef-4a63-9f07-c6aa1a575385 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:04:41.191884 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591da9f7-3fef-4a63-9f07-c6aa1a575385 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:04:41.200705 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591da9f7-3fef-4a63-9f07-c6aa1a575385 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:04:41.202114 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591da9f7-3fef-4a63-9f07-c6aa1a575385 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:04:41.211918 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 765/1533] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:04:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:04:44.608074 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878775c-fa77-4f9f-bccd-4e5b8d5c93d0 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:04:44.608701 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878775c-fa77-4f9f-bccd-4e5b8d5c93d0 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:04:44.608701 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878775c-fa77-4f9f-bccd-4e5b8d5c93d0 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:04:44.609982 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 4 01:04:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:04:44.615039 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17437c5a-e00d-418a-91a9-88d658fd1c61 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:04:44.615989 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17437c5a-e00d-418a-91a9-88d658fd1c61 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:04:44.616188 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17437c5a-e00d-418a-91a9-88d658fd1c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:04:44.635338 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-17437c5a-e00d-418a-91a9-88d658fd1c61 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:04:44.636572 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-17437c5a-e00d-418a-91a9-88d658fd1c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:04:44.644586 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 4 01:04:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:04:45.199267 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5878775c-fa77-4f9f-bccd-4e5b8d5c93d0 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:04:45.200973 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5878775c-fa77-4f9f-bccd-4e5b8d5c93d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:04:45.266055 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:04:45.279763 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5878775c-fa77-4f9f-bccd-4e5b8d5c93d0 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:04:45.287224 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c48e6-546a-47d0-8b99-dc9cab554564 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:04:45.287224 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c48e6-546a-47d0-8b99-dc9cab554564 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:04:45.287224 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c48e6-546a-47d0-8b99-dc9cab554564 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:04:45.287224 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c48e6-546a-47d0-8b99-dc9cab554564 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:04:45.293611 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c48e6-546a-47d0-8b99-dc9cab554564 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:04:45.295033 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c48e6-546a-47d0-8b99-dc9cab554564 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:04:45.304291 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 770/1536] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:04:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:04:47.207996 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab41daa-7f63-47b5-88ce-3df0449ac3ff None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:04:47.208592 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab41daa-7f63-47b5-88ce-3df0449ac3ff None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:04:47.208592 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab41daa-7f63-47b5-88ce-3df0449ac3ff None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:04:47.210100 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 4 01:04:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:04:47.215162 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423f012-234c-4fea-b559-b378764c6b49 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:04:47.215578 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423f012-234c-4fea-b559-b378764c6b49 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:04:47.215711 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423f012-234c-4fea-b559-b378764c6b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:04:47.235147 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-e423f012-234c-4fea-b559-b378764c6b49 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:04:47.236352 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e423f012-234c-4fea-b559-b378764c6b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:04:47.244799 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 4 01:04:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:04:47.805206 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab41daa-7f63-47b5-88ce-3df0449ac3ff None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:04:47.806901 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-eab41daa-7f63-47b5-88ce-3df0449ac3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:04:47.819937 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab41daa-7f63-47b5-88ce-3df0449ac3ff None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:04:47.823459 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a38c62-460d-41cc-887d-a4cc4e2b2154 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:04:47.824046 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a38c62-460d-41cc-887d-a4cc4e2b2154 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:04:47.824324 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a38c62-460d-41cc-887d-a4cc4e2b2154 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:04:47.824629 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a38c62-460d-41cc-887d-a4cc4e2b2154 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:04:47.833321 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a38c62-460d-41cc-887d-a4cc4e2b2154 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:04:47.835571 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a38c62-460d-41cc-887d-a4cc4e2b2154 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:04:47.844701 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 768/1539] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:04:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:04:57.230602 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=11 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:04:57.840674 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=33 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:22.624652 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ba31b-ceb6-4a36-960f-aff9bd9a1be7 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:22.638315 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ba31b-ceb6-4a36-960f-aff9bd9a1be7 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:22.638315 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ba31b-ceb6-4a36-960f-aff9bd9a1be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:22.653871 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2389a-c2b9-4c9b-b6c5-801ecaef2f84 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:22.654456 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2389a-c2b9-4c9b-b6c5-801ecaef2f84 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:22.654456 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2389a-c2b9-4c9b-b6c5-801ecaef2f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:22.657455 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-040ba31b-ceb6-4a36-960f-aff9bd9a1be7 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:22.658678 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-040ba31b-ceb6-4a36-960f-aff9bd9a1be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:22.673195 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 772/1540] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:06:22.683665 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5dd2389a-c2b9-4c9b-b6c5-801ecaef2f84 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:22.684970 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5dd2389a-c2b9-4c9b-b6c5-801ecaef2f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:22.696416 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf0a8c-0b23-4a5b-b43e-83fb69fd8f5b None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:22.717878 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf0a8c-0b23-4a5b-b43e-83fb69fd8f5b None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:22.717878 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf0a8c-0b23-4a5b-b43e-83fb69fd8f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:22.718166 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 769/1541] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:22.718166 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb235f-7a94-4215-a9f8-ba83498f2f90 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:22.718166 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb235f-7a94-4215-a9f8-ba83498f2f90 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:22.718166 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb235f-7a94-4215-a9f8-ba83498f2f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:22.739825 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-dbdf0a8c-0b23-4a5b-b43e-83fb69fd8f5b None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:22.744937 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-dbdf0a8c-0b23-4a5b-b43e-83fb69fd8f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:22.745117 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-d8cb235f-7a94-4215-a9f8-ba83498f2f90 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:22.745117 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d8cb235f-7a94-4215-a9f8-ba83498f2f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:22.754558 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 770/1542] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:22.759764 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb235f-7a94-4215-a9f8-ba83498f2f90 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:22.767175 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 773/1543] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 04 01:06:22.775071 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb235f-7a94-4215-a9f8-ba83498f2f90 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:22.778732 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab1abb-287a-462e-8f34-390a87ef0987 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:22.779021 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab1abb-287a-462e-8f34-390a87ef0987 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:22.779296 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab1abb-287a-462e-8f34-390a87ef0987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:22.783071 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c3e1ea-0add-482e-bcd6-e7bbc3937bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:22.783701 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3e1ea-0add-482e-bcd6-e7bbc3937bd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:22.783980 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3e1ea-0add-482e-bcd6-e7bbc3937bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:22.785031 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3e1ea-0add-482e-bcd6-e7bbc3937bd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:22.785730 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c3e1ea-0add-482e-bcd6-e7bbc3937bd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:06:22.786034 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c3e1ea-0add-482e-bcd6-e7bbc3937bd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:22.787048 dv-cinder-master-3-smb devstack@keystone.service[81629]: WARNING py.warnings [None req-44c3e1ea-0add-482e-bcd6-e7bbc3937bd3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 04 01:06:22.787048 dv-cinder-master-3-smb devstack@keystone.service[81629]: warnings.warn(msg) Sep 04 01:06:22.787048 dv-cinder-master-3-smb devstack@keystone.service[81629]: Sep 04 01:06:22.787984 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c3e1ea-0add-482e-bcd6-e7bbc3937bd3 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:22.808378 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 771/1544] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 01:06:22.809434 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c3e1ea-0add-482e-bcd6-e7bbc3937bd3 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:22.820906 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c3e1ea-0add-482e-bcd6-e7bbc3937bd3 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:22.824411 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-6eab1abb-287a-462e-8f34-390a87ef0987 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:22.825680 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-6eab1abb-287a-462e-8f34-390a87ef0987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:22.830969 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b2775-31b3-40e1-a4a7-1ccae6e16417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:22.832351 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b2775-31b3-40e1-a4a7-1ccae6e16417 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:22.832707 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b2775-31b3-40e1-a4a7-1ccae6e16417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:22.832989 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b2775-31b3-40e1-a4a7-1ccae6e16417 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:22.833645 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b2775-31b3-40e1-a4a7-1ccae6e16417 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:06:22.833951 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b2775-31b3-40e1-a4a7-1ccae6e16417 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:22.835545 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b2775-31b3-40e1-a4a7-1ccae6e16417 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:22.844385 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 774/1545] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:22.845834 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981dbcd1-5f0e-4531-83ab-c901516ae738 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:22.846395 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981dbcd1-5f0e-4531-83ab-c901516ae738 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:22.846676 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981dbcd1-5f0e-4531-83ab-c901516ae738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:22.854189 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 772/1546] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:06:22.855026 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b2775-31b3-40e1-a4a7-1ccae6e16417 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:22.866056 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b2775-31b3-40e1-a4a7-1ccae6e16417 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:22.870773 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6039017-8a01-42d0-8824-f1576731f05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:22.871368 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6039017-8a01-42d0-8824-f1576731f05c admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:22.871648 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6039017-8a01-42d0-8824-f1576731f05c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:22.871899 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-981dbcd1-5f0e-4531-83ab-c901516ae738 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:22.872014 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6039017-8a01-42d0-8824-f1576731f05c admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:22.872568 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6039017-8a01-42d0-8824-f1576731f05c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:22.873137 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-981dbcd1-5f0e-4531-83ab-c901516ae738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:22.874462 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6039017-8a01-42d0-8824-f1576731f05c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:22.881580 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 775/1547] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:22.882694 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8a891-78c5-41af-8b19-a28b734977fd None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:22.882975 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8a891-78c5-41af-8b19-a28b734977fd None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:22.883242 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8a891-78c5-41af-8b19-a28b734977fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:22.906129 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-a7c8a891-78c5-41af-8b19-a28b734977fd None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:22.907326 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a7c8a891-78c5-41af-8b19-a28b734977fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:22.915506 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 776/1548] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:22.916693 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac8099-3611-4e5d-b5e2-2080ede63b85 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:22.916974 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac8099-3611-4e5d-b5e2-2080ede63b85 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:22.917241 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac8099-3611-4e5d-b5e2-2080ede63b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:22.936062 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-26ac8099-3611-4e5d-b5e2-2080ede63b85 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:22.941855 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-26ac8099-3611-4e5d-b5e2-2080ede63b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:22.945654 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 777/1549] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:22.946784 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ace8fe-c6b2-4d4f-b7e2-8820eacb63f2 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:22.947070 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ace8fe-c6b2-4d4f-b7e2-8820eacb63f2 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:22.947386 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ace8fe-c6b2-4d4f-b7e2-8820eacb63f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:22.966893 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-09ace8fe-c6b2-4d4f-b7e2-8820eacb63f2 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:22.970756 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 773/1550] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:06:22 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:22.970901 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-09ace8fe-c6b2-4d4f-b7e2-8820eacb63f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:22.975843 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f08dc0-667d-46ca-a92d-62ef111a6df9 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:22.975843 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f08dc0-667d-46ca-a92d-62ef111a6df9 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:22.975843 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f08dc0-667d-46ca-a92d-62ef111a6df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:22.989120 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-e9f08dc0-667d-46ca-a92d-62ef111a6df9 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:22.990313 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e9f08dc0-667d-46ca-a92d-62ef111a6df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:23.030546 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 778/1551] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:23.031573 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fb216-02c9-449b-b364-b9daf312a59a None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:23.031853 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fb216-02c9-449b-b364-b9daf312a59a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:23.032129 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fb216-02c9-449b-b364-b9daf312a59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:23.051548 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-700fb216-02c9-449b-b364-b9daf312a59a None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:23.059927 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 774/1552] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:23.060097 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-700fb216-02c9-449b-b364-b9daf312a59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:23.060209 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f08dc0-667d-46ca-a92d-62ef111a6df9 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:23.062594 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f08dc0-667d-46ca-a92d-62ef111a6df9 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:23.068078 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21af2b5-4eb6-46b8-b9d5-bbb365188f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:23.068268 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 779/1553] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:23.068699 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21af2b5-4eb6-46b8-b9d5-bbb365188f46 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:23.068975 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21af2b5-4eb6-46b8-b9d5-bbb365188f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:23.069244 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21af2b5-4eb6-46b8-b9d5-bbb365188f46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:23.069559 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d59419-ee29-4f6d-900f-3d372d13efc3 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:23.069851 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d59419-ee29-4f6d-900f-3d372d13efc3 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:23.069962 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21af2b5-4eb6-46b8-b9d5-bbb365188f46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:23.070121 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d59419-ee29-4f6d-900f-3d372d13efc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:23.071746 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21af2b5-4eb6-46b8-b9d5-bbb365188f46 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:23.090509 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-46d59419-ee29-4f6d-900f-3d372d13efc3 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:23.091720 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-46d59419-ee29-4f6d-900f-3d372d13efc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:23.100625 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 780/1554] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:23.103279 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edff931-3c7d-4b1e-a4aa-f6d1495992b0 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:23.103279 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edff931-3c7d-4b1e-a4aa-f6d1495992b0 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:23.103279 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edff931-3c7d-4b1e-a4aa-f6d1495992b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:23.121644 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-0edff931-3c7d-4b1e-a4aa-f6d1495992b0 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:23.125216 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-0edff931-3c7d-4b1e-a4aa-f6d1495992b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:23.130628 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 781/1555] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:23.131814 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09ef49-031f-44ff-af36-159087a4c94b None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:23.132039 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09ef49-031f-44ff-af36-159087a4c94b None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:23.132310 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09ef49-031f-44ff-af36-159087a4c94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:23.151680 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-7f09ef49-031f-44ff-af36-159087a4c94b None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:23.153905 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-7f09ef49-031f-44ff-af36-159087a4c94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:23.161282 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 782/1556] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:23.162013 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f09ef49-031f-44ff-af36-159087a4c94b None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:23.171172 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f09ef49-031f-44ff-af36-159087a4c94b None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:23.180522 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a5acd-9caa-4443-bda1-6965633ecd07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:23.180522 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a5acd-9caa-4443-bda1-6965633ecd07 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:23.180522 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a5acd-9caa-4443-bda1-6965633ecd07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:23.180522 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a5acd-9caa-4443-bda1-6965633ecd07 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:23.180936 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67a5acd-9caa-4443-bda1-6965633ecd07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:23.180936 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67a5acd-9caa-4443-bda1-6965633ecd07 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:23.245259 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 775/1557] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:06:23 2022] POST /identity/v3/projects => generated 389 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:23.246263 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21af2b5-4eb6-46b8-b9d5-bbb365188f46 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:23.270384 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 783/1558] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:06:23 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:23.274409 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a2bda-b992-44ee-8eb6-07ace5917479 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:23.274409 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a2bda-b992-44ee-8eb6-07ace5917479 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:23.274409 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a2bda-b992-44ee-8eb6-07ace5917479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:23.292506 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-cb0a2bda-b992-44ee-8eb6-07ace5917479 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:23.293719 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-cb0a2bda-b992-44ee-8eb6-07ace5917479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:23.309914 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21af2b5-4eb6-46b8-b9d5-bbb365188f46 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:23.314624 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a64b5c-ae72-4ecb-b583-315b513223ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:23.315224 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a64b5c-ae72-4ecb-b583-315b513223ef admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:23.315505 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a64b5c-ae72-4ecb-b583-315b513223ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:23.315789 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a64b5c-ae72-4ecb-b583-315b513223ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:23.316856 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a64b5c-ae72-4ecb-b583-315b513223ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:06:23.317163 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a64b5c-ae72-4ecb-b583-315b513223ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:23.318753 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a64b5c-ae72-4ecb-b583-315b513223ef admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:23.330105 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 776/1559] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:06:23.331927 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0712b1-506f-4a53-9db5-d7ff07eead83 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:23.332445 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0712b1-506f-4a53-9db5-d7ff07eead83 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:23.333051 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0712b1-506f-4a53-9db5-d7ff07eead83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:23.341204 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:23.353895 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-7e0712b1-506f-4a53-9db5-d7ff07eead83 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:23.355119 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7e0712b1-506f-4a53-9db5-d7ff07eead83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:23.355434 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 784/1560] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:23.356286 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a2bda-b992-44ee-8eb6-07ace5917479 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:23.364132 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 777/1561] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:23.365121 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0712b1-506f-4a53-9db5-d7ff07eead83 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:23.368937 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a2bda-b992-44ee-8eb6-07ace5917479 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:23.374647 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b087b0-449b-41b8-a1db-f2973afd0214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:23.375282 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b087b0-449b-41b8-a1db-f2973afd0214 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:23.375493 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0712b1-506f-4a53-9db5-d7ff07eead83 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:23.375646 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b087b0-449b-41b8-a1db-f2973afd0214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:23.375855 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b087b0-449b-41b8-a1db-f2973afd0214 admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:23.376529 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b087b0-449b-41b8-a1db-f2973afd0214 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:06:23.376840 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b087b0-449b-41b8-a1db-f2973afd0214 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:23.379043 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b087b0-449b-41b8-a1db-f2973afd0214 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:23.380883 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ddc15-0765-4a5b-8403-d16b24c149df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:23.381473 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ddc15-0765-4a5b-8403-d16b24c149df admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:23.381758 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ddc15-0765-4a5b-8403-d16b24c149df admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:23.393030 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ddc15-0765-4a5b-8403-d16b24c149df admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:23.393030 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390ddc15-0765-4a5b-8403-d16b24c149df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:06:23.393030 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390ddc15-0765-4a5b-8403-d16b24c149df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:23.393030 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390ddc15-0765-4a5b-8403-d16b24c149df admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:23.393413 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 785/1562] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:06:23.393413 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b087b0-449b-41b8-a1db-f2973afd0214 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:23.399515 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:23.401741 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 778/1563] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:06:23.402515 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ddc15-0765-4a5b-8403-d16b24c149df admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:23.403265 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b087b0-449b-41b8-a1db-f2973afd0214 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:23.407920 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291fd89f-f1fc-41a8-b1c6-464397bc81bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:23.408773 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291fd89f-f1fc-41a8-b1c6-464397bc81bb admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:23.409060 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291fd89f-f1fc-41a8-b1c6-464397bc81bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:23.409336 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291fd89f-f1fc-41a8-b1c6-464397bc81bb admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:23.410374 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291fd89f-f1fc-41a8-b1c6-464397bc81bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:06:23.410703 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291fd89f-f1fc-41a8-b1c6-464397bc81bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:23.412302 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291fd89f-f1fc-41a8-b1c6-464397bc81bb admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:23.413098 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ddc15-0765-4a5b-8403-d16b24c149df admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:23.420602 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59743d-0169-4650-a7f3-2b4a75a5346e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:23.420602 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59743d-0169-4650-a7f3-2b4a75a5346e admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:23.420602 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59743d-0169-4650-a7f3-2b4a75a5346e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:23.420602 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59743d-0169-4650-a7f3-2b4a75a5346e admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:23.421003 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb59743d-0169-4650-a7f3-2b4a75a5346e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:23.421003 dv-cinder-master-3-smb devstack@keystone.service[81629]: WARNING py.warnings [None req-cb59743d-0169-4650-a7f3-2b4a75a5346e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 04 01:06:23.421003 dv-cinder-master-3-smb devstack@keystone.service[81629]: warnings.warn(msg) Sep 04 01:06:23.421003 dv-cinder-master-3-smb devstack@keystone.service[81629]: Sep 04 01:06:23.421809 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb59743d-0169-4650-a7f3-2b4a75a5346e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:23.424714 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 786/1564] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:06:23.425741 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291fd89f-f1fc-41a8-b1c6-464397bc81bb admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:23.435436 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291fd89f-f1fc-41a8-b1c6-464397bc81bb admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:23.440107 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e192e-0ef2-4bb9-a290-4bf1f72ce192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:23.441261 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e192e-0ef2-4bb9-a290-4bf1f72ce192 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:23.441762 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e192e-0ef2-4bb9-a290-4bf1f72ce192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:23.443617 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e192e-0ef2-4bb9-a290-4bf1f72ce192 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:23.443617 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e192e-0ef2-4bb9-a290-4bf1f72ce192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:23.444390 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e192e-0ef2-4bb9-a290-4bf1f72ce192 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:23.454187 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 787/1565] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:23.455059 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e192e-0ef2-4bb9-a290-4bf1f72ce192 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:23.465404 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e192e-0ef2-4bb9-a290-4bf1f72ce192 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:23.470402 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21acc801-7a26-4726-933d-21f4d7212ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:23.470973 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21acc801-7a26-4726-933d-21f4d7212ad0 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:23.471257 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21acc801-7a26-4726-933d-21f4d7212ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:23.472344 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21acc801-7a26-4726-933d-21f4d7212ad0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:23.472492 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21acc801-7a26-4726-933d-21f4d7212ad0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:23.473510 dv-cinder-master-3-smb devstack@keystone.service[81630]: WARNING py.warnings [None req-21acc801-7a26-4726-933d-21f4d7212ad0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 04 01:06:23.473510 dv-cinder-master-3-smb devstack@keystone.service[81630]: warnings.warn(msg) Sep 04 01:06:23.473510 dv-cinder-master-3-smb devstack@keystone.service[81630]: Sep 04 01:06:23.474695 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21acc801-7a26-4726-933d-21f4d7212ad0 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:23.498652 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 779/1566] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:06:23 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:23.499496 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59743d-0169-4650-a7f3-2b4a75a5346e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:23.509614 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59743d-0169-4650-a7f3-2b4a75a5346e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:23.514466 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40d05b-bc25-4122-a685-3e8ac49b6521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:23.515346 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40d05b-bc25-4122-a685-3e8ac49b6521 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:23.515631 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40d05b-bc25-4122-a685-3e8ac49b6521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:23.515910 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40d05b-bc25-4122-a685-3e8ac49b6521 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:23.516584 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de40d05b-bc25-4122-a685-3e8ac49b6521 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:06:23.516895 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de40d05b-bc25-4122-a685-3e8ac49b6521 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:23.518478 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de40d05b-bc25-4122-a685-3e8ac49b6521 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:23.530896 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 780/1567] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:06:23.531696 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40d05b-bc25-4122-a685-3e8ac49b6521 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:23.542059 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40d05b-bc25-4122-a685-3e8ac49b6521 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:23.546827 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f05818-842c-4ab2-97e2-61b209c96c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:23.547403 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f05818-842c-4ab2-97e2-61b209c96c85 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:23.547774 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f05818-842c-4ab2-97e2-61b209c96c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:23.548048 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f05818-842c-4ab2-97e2-61b209c96c85 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:23.548722 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f05818-842c-4ab2-97e2-61b209c96c85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:06:23.549028 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f05818-842c-4ab2-97e2-61b209c96c85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:23.550706 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f05818-842c-4ab2-97e2-61b209c96c85 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:23.563402 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 781/1568] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:06:23.564171 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f05818-842c-4ab2-97e2-61b209c96c85 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:23.574139 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f05818-842c-4ab2-97e2-61b209c96c85 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:23.578908 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4740a2-ca4f-4d01-8045-c4f771533ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:23.579476 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4740a2-ca4f-4d01-8045-c4f771533ecb admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:23.579759 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4740a2-ca4f-4d01-8045-c4f771533ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:23.580033 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4740a2-ca4f-4d01-8045-c4f771533ecb admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:23.580685 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4740a2-ca4f-4d01-8045-c4f771533ecb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:23.583246 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4740a2-ca4f-4d01-8045-c4f771533ecb admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:23.597249 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 788/1569] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:06:23 2022] POST /identity/v3/projects => generated 377 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:23.598437 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21acc801-7a26-4726-933d-21f4d7212ad0 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:23.658993 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21acc801-7a26-4726-933d-21f4d7212ad0 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:23.663759 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a2e5d-372c-40ce-9418-3f438e0be782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:23.664360 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a2e5d-372c-40ce-9418-3f438e0be782 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:23.664676 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a2e5d-372c-40ce-9418-3f438e0be782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:23.664951 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a2e5d-372c-40ce-9418-3f438e0be782 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:23.667078 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8a2e5d-372c-40ce-9418-3f438e0be782 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:23.667510 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8a2e5d-372c-40ce-9418-3f438e0be782 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:23.679762 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 782/1570] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:06:23 2022] POST /identity/v3/projects => generated 401 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:23.680432 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4740a2-ca4f-4d01-8045-c4f771533ecb admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:23.743916 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4740a2-ca4f-4d01-8045-c4f771533ecb admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:23.748712 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d754f-c736-4401-85a6-e84409021d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:23.749311 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d754f-c736-4401-85a6-e84409021d56 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:23.749593 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d754f-c736-4401-85a6-e84409021d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:23.749860 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d754f-c736-4401-85a6-e84409021d56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:23.750557 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1d754f-c736-4401-85a6-e84409021d56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:23.752539 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1d754f-c736-4401-85a6-e84409021d56 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:23.759704 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 789/1571] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:06:23 2022] POST /identity/v3/projects => generated 391 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:23.760663 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a2e5d-372c-40ce-9418-3f438e0be782 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:23.826658 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a2e5d-372c-40ce-9418-3f438e0be782 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:23.831941 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1331e8-46e1-415a-b128-bb6d00224a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:23.832576 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1331e8-46e1-415a-b128-bb6d00224a14 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:23.832976 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1331e8-46e1-415a-b128-bb6d00224a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:23.833259 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1331e8-46e1-415a-b128-bb6d00224a14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:23.833899 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1331e8-46e1-415a-b128-bb6d00224a14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:23.835574 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1331e8-46e1-415a-b128-bb6d00224a14 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:23.846033 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 783/1572] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:06:23 2022] POST /identity/v3/projects => generated 385 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:23.849298 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d754f-c736-4401-85a6-e84409021d56 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:23.859112 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 790/1573] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:23.859112 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1331e8-46e1-415a-b128-bb6d00224a14 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:23.859112 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1331e8-46e1-415a-b128-bb6d00224a14 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:23.903100 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:23.917687 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d754f-c736-4401-85a6-e84409021d56 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:23.920924 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce3296a-8259-443c-8e7d-01773e80f796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:23.921111 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c54aa-ef99-4480-b478-65b978683f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:23.921535 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3296a-8259-443c-8e7d-01773e80f796 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:23.921710 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c54aa-ef99-4480-b478-65b978683f7b admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:23.921811 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3296a-8259-443c-8e7d-01773e80f796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:23.921987 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c54aa-ef99-4480-b478-65b978683f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:23.922090 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3296a-8259-443c-8e7d-01773e80f796 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:23.922256 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c54aa-ef99-4480-b478-65b978683f7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:23.922692 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce3296a-8259-443c-8e7d-01773e80f796 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:23.922884 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61c54aa-ef99-4480-b478-65b978683f7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:23.924477 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61c54aa-ef99-4480-b478-65b978683f7b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:23.924698 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce3296a-8259-443c-8e7d-01773e80f796 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:23.927078 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:23.938462 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 784/1574] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:23.938462 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c54aa-ef99-4480-b478-65b978683f7b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:23.950508 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c54aa-ef99-4480-b478-65b978683f7b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:23.955120 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0d9886-3aee-4cd1-a2b8-2c7f1a594e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:23.955707 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d9886-3aee-4cd1-a2b8-2c7f1a594e64 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:23.955984 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d9886-3aee-4cd1-a2b8-2c7f1a594e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:23.956307 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d9886-3aee-4cd1-a2b8-2c7f1a594e64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:23.957029 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d9886-3aee-4cd1-a2b8-2c7f1a594e64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:23.958908 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d9886-3aee-4cd1-a2b8-2c7f1a594e64 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:24.053893 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 791/1575] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:06:23 2022] POST /identity/v3/projects => generated 383 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:24.054723 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce3296a-8259-443c-8e7d-01773e80f796 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:24.087918 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 785/1576] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:06:23 2022] POST /identity/v3/projects => generated 365 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:24.090933 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0d9886-3aee-4cd1-a2b8-2c7f1a594e64 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:24.116648 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce3296a-8259-443c-8e7d-01773e80f796 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:24.151897 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0d9886-3aee-4cd1-a2b8-2c7f1a594e64 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:24.156553 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ccbf9c-7f07-482b-bf3e-015611fc1829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:24.157155 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ccbf9c-7f07-482b-bf3e-015611fc1829 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:24.157440 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ccbf9c-7f07-482b-bf3e-015611fc1829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:24.157714 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ccbf9c-7f07-482b-bf3e-015611fc1829 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:24.158364 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ccbf9c-7f07-482b-bf3e-015611fc1829 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:24.161080 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ccbf9c-7f07-482b-bf3e-015611fc1829 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:24.174057 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a5362-fdcb-4edd-9b0b-6b2a831ffdae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:24.174615 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a5362-fdcb-4edd-9b0b-6b2a831ffdae admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:24.174881 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a5362-fdcb-4edd-9b0b-6b2a831ffdae admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:24.175731 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a5362-fdcb-4edd-9b0b-6b2a831ffdae admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:24.176622 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a5362-fdcb-4edd-9b0b-6b2a831ffdae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:24.178491 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a5362-fdcb-4edd-9b0b-6b2a831ffdae admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:24.280644 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 786/1577] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:06:24 2022] POST /identity/v3/users => generated 301 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:24.281638 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ccbf9c-7f07-482b-bf3e-015611fc1829 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:24.292182 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ccbf9c-7f07-482b-bf3e-015611fc1829 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:24.296813 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4241952-162f-4923-8654-1ec920bee985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:24.297417 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4241952-162f-4923-8654-1ec920bee985 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:24.297697 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4241952-162f-4923-8654-1ec920bee985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:24.297969 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4241952-162f-4923-8654-1ec920bee985 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:24.298581 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4241952-162f-4923-8654-1ec920bee985 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:24.301117 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4241952-162f-4923-8654-1ec920bee985 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:24.309376 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 792/1578] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:06:24 2022] POST /identity/v3/users => generated 289 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:24.316471 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a5362-fdcb-4edd-9b0b-6b2a831ffdae admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:24.320226 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a5362-fdcb-4edd-9b0b-6b2a831ffdae admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:24.324851 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70234c3c-77f4-47f3-8ed9-854cc7a128b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:24.325478 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70234c3c-77f4-47f3-8ed9-854cc7a128b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:24.325761 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70234c3c-77f4-47f3-8ed9-854cc7a128b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:24.326035 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70234c3c-77f4-47f3-8ed9-854cc7a128b9 admin admin] PATH_INFO: `/v3/projects/ef5de4321f9c4687a0bae83183b59a39/users/0c5e9731f6204b44841b8524762d5a77/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:24.340374 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70234c3c-77f4-47f3-8ed9-854cc7a128b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef5de4321f9c4687a0bae83183b59a39, user_id=0c5e9731f6204b44841b8524762d5a77, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:24.352052 dv-cinder-master-3-smb devstack@keystone.service[81630]: WARNING py.warnings [None req-70234c3c-77f4-47f3-8ed9-854cc7a128b9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 04 01:06:24.352052 dv-cinder-master-3-smb devstack@keystone.service[81630]: warnings.warn(msg) Sep 04 01:06:24.352052 dv-cinder-master-3-smb devstack@keystone.service[81630]: Sep 04 01:06:24.352052 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70234c3c-77f4-47f3-8ed9-854cc7a128b9 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:24.406041 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 787/1579] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:06:24 2022] POST /identity/v3/users => generated 296 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:24.407008 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4241952-162f-4923-8654-1ec920bee985 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:24.417012 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4241952-162f-4923-8654-1ec920bee985 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:24.421644 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c54e05-f828-4202-bef7-1b4721f3640b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:24.422212 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c54e05-f828-4202-bef7-1b4721f3640b admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:24.422489 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c54e05-f828-4202-bef7-1b4721f3640b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:24.422798 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c54e05-f828-4202-bef7-1b4721f3640b admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:24.423405 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c54e05-f828-4202-bef7-1b4721f3640b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:24.426006 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c54e05-f828-4202-bef7-1b4721f3640b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:24.435452 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 793/1580] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:06:24 2022] PUT /identity/v3/projects/ef5de4321f9c4687a0bae83183b59a39/users/0c5e9731f6204b44841b8524762d5a77/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:24.438678 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70234c3c-77f4-47f3-8ed9-854cc7a128b9 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:24.477438 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:24.497809 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70234c3c-77f4-47f3-8ed9-854cc7a128b9 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:24.502414 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508091a2-ffb8-4bcb-ba28-1cc5332e893f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:24.502997 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508091a2-ffb8-4bcb-ba28-1cc5332e893f admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:24.503443 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508091a2-ffb8-4bcb-ba28-1cc5332e893f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:24.503669 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508091a2-ffb8-4bcb-ba28-1cc5332e893f admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:24.504320 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508091a2-ffb8-4bcb-ba28-1cc5332e893f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:24.505935 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508091a2-ffb8-4bcb-ba28-1cc5332e893f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:24.515746 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 794/1581] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:24.516524 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508091a2-ffb8-4bcb-ba28-1cc5332e893f admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:24.525778 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 788/1582] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:06:24 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:24.526487 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c54e05-f828-4202-bef7-1b4721f3640b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:24.526602 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508091a2-ffb8-4bcb-ba28-1cc5332e893f admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:24.533353 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0f4c17-da06-4f35-9233-7f10e0842ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:24.534033 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f4c17-da06-4f35-9233-7f10e0842ada admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:24.535558 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f4c17-da06-4f35-9233-7f10e0842ada admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:24.535864 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f4c17-da06-4f35-9233-7f10e0842ada admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:24.536524 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0f4c17-da06-4f35-9233-7f10e0842ada admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:24.538449 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0f4c17-da06-4f35-9233-7f10e0842ada admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:24.539240 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c54e05-f828-4202-bef7-1b4721f3640b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:24.544004 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375a2bc-6418-49cd-86b9-6c0902a69bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:24.544637 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375a2bc-6418-49cd-86b9-6c0902a69bf3 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:24.544937 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375a2bc-6418-49cd-86b9-6c0902a69bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:24.545194 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375a2bc-6418-49cd-86b9-6c0902a69bf3 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:24.545818 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6375a2bc-6418-49cd-86b9-6c0902a69bf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:24.547781 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6375a2bc-6418-49cd-86b9-6c0902a69bf3 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:24.618029 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 795/1583] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:06:24 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:24.618871 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0f4c17-da06-4f35-9233-7f10e0842ada admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:24.621577 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 789/1584] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:06:24 2022] POST /identity/v3/users => generated 283 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:24.622525 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375a2bc-6418-49cd-86b9-6c0902a69bf3 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:24.631104 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0f4c17-da06-4f35-9233-7f10e0842ada admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:24.634155 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375a2bc-6418-49cd-86b9-6c0902a69bf3 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:24.636016 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c5032-b949-4d25-bd93-f4b7498ae306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:24.636659 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c5032-b949-4d25-bd93-f4b7498ae306 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:24.636948 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c5032-b949-4d25-bd93-f4b7498ae306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:24.637226 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c5032-b949-4d25-bd93-f4b7498ae306 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:24.637893 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c5032-b949-4d25-bd93-f4b7498ae306 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:24.639506 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c5032-b949-4d25-bd93-f4b7498ae306 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:24.640273 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a55d8a-2506-4a93-ae35-2de8939b1641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:24.641088 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a55d8a-2506-4a93-ae35-2de8939b1641 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:24.641569 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a55d8a-2506-4a93-ae35-2de8939b1641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:24.642954 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a55d8a-2506-4a93-ae35-2de8939b1641 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:24.642954 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a55d8a-2506-4a93-ae35-2de8939b1641 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:24.645375 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a55d8a-2506-4a93-ae35-2de8939b1641 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:24.651985 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 796/1585] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:24.652852 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c5032-b949-4d25-bd93-f4b7498ae306 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:24.661010 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 790/1586] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:24.662444 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0cab71-699e-4488-91db-80233a857896 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:24.662729 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0cab71-699e-4488-91db-80233a857896 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:24.663006 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0cab71-699e-4488-91db-80233a857896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:24.666554 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c5032-b949-4d25-bd93-f4b7498ae306 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:24.671325 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31baf02e-9d5e-4187-82a5-432cfe52a11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:24.671927 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31baf02e-9d5e-4187-82a5-432cfe52a11e admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:24.672211 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31baf02e-9d5e-4187-82a5-432cfe52a11e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:24.672516 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31baf02e-9d5e-4187-82a5-432cfe52a11e admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:24.673155 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31baf02e-9d5e-4187-82a5-432cfe52a11e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:24.674825 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31baf02e-9d5e-4187-82a5-432cfe52a11e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:24.681890 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-3e0cab71-699e-4488-91db-80233a857896 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:24.683086 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3e0cab71-699e-4488-91db-80233a857896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:24.686204 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 797/1587] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:24.686987 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31baf02e-9d5e-4187-82a5-432cfe52a11e admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:24.693708 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 791/1588] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:24.694566 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0cab71-699e-4488-91db-80233a857896 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:24.699105 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31baf02e-9d5e-4187-82a5-432cfe52a11e admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:24.703873 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c497d-121f-4247-a335-9a69c2bd529d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:24.704549 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c497d-121f-4247-a335-9a69c2bd529d admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:24.704834 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c497d-121f-4247-a335-9a69c2bd529d admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:24.705111 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c497d-121f-4247-a335-9a69c2bd529d admin admin] PATH_INFO: `/v3/projects/e334b7aa2f1848c1999d86ac666fe65e/users/bd1c41fcdc724a128117a5288b8559f5/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:24.706775 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0cab71-699e-4488-91db-80233a857896 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:24.711561 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0221e90-e59d-4593-a578-149ab659af33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:24.712178 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0221e90-e59d-4593-a578-149ab659af33 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:24.712436 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0221e90-e59d-4593-a578-149ab659af33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:24.712735 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0221e90-e59d-4593-a578-149ab659af33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:24.713374 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0221e90-e59d-4593-a578-149ab659af33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:24.714941 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0221e90-e59d-4593-a578-149ab659af33 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:24.720044 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c497d-121f-4247-a335-9a69c2bd529d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e334b7aa2f1848c1999d86ac666fe65e, user_id=bd1c41fcdc724a128117a5288b8559f5, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:24.723336 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c497d-121f-4247-a335-9a69c2bd529d admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:24.728833 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 792/1589] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:24.728833 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0221e90-e59d-4593-a578-149ab659af33 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:24.736174 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0221e90-e59d-4593-a578-149ab659af33 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:24.740873 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10905018-3638-45a8-b9c0-ccc2464fc7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:24.741436 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10905018-3638-45a8-b9c0-ccc2464fc7e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:24.745878 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10905018-3638-45a8-b9c0-ccc2464fc7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:24.745878 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10905018-3638-45a8-b9c0-ccc2464fc7e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:24.745878 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10905018-3638-45a8-b9c0-ccc2464fc7e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:24.745878 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10905018-3638-45a8-b9c0-ccc2464fc7e9 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:24.754046 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 793/1590] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:24.754823 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10905018-3638-45a8-b9c0-ccc2464fc7e9 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:24.764357 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10905018-3638-45a8-b9c0-ccc2464fc7e9 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:24.769830 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8027e417-103a-441b-817a-2e19ceb2c925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:24.770371 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027e417-103a-441b-817a-2e19ceb2c925 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:24.770651 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027e417-103a-441b-817a-2e19ceb2c925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:24.770954 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027e417-103a-441b-817a-2e19ceb2c925 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:24.771568 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8027e417-103a-441b-817a-2e19ceb2c925 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:24.773151 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8027e417-103a-441b-817a-2e19ceb2c925 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:24.783339 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 794/1591] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:24.785657 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8027e417-103a-441b-817a-2e19ceb2c925 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:24.795633 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8027e417-103a-441b-817a-2e19ceb2c925 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:24.800592 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87233016-7187-42d5-9833-01fe4a70a309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:24.801163 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87233016-7187-42d5-9833-01fe4a70a309 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:24.801450 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87233016-7187-42d5-9833-01fe4a70a309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:24.801727 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87233016-7187-42d5-9833-01fe4a70a309 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:24.802352 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87233016-7187-42d5-9833-01fe4a70a309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:24.803492 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 798/1592] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:06:24 2022] PUT /identity/v3/projects/e334b7aa2f1848c1999d86ac666fe65e/users/bd1c41fcdc724a128117a5288b8559f5/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:24.803913 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87233016-7187-42d5-9833-01fe4a70a309 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:24.804317 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c497d-121f-4247-a335-9a69c2bd529d admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:24.817684 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 795/1593] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:24.817684 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87233016-7187-42d5-9833-01fe4a70a309 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:24.824846 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87233016-7187-42d5-9833-01fe4a70a309 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:24.833027 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:24.875052 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c497d-121f-4247-a335-9a69c2bd529d admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:24.880614 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b5161-63eb-452c-9f2b-2a2849e0e3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:24.881230 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b5161-63eb-452c-9f2b-2a2849e0e3ba admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:24.881516 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b5161-63eb-452c-9f2b-2a2849e0e3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:24.881816 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b5161-63eb-452c-9f2b-2a2849e0e3ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:24.882456 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b5161-63eb-452c-9f2b-2a2849e0e3ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:24.883999 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f1443-44c6-4b60-8260-8efb86e74c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:24.884140 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b5161-63eb-452c-9f2b-2a2849e0e3ba admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:24.884649 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f1443-44c6-4b60-8260-8efb86e74c4e admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:24.884932 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f1443-44c6-4b60-8260-8efb86e74c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:24.885378 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f1443-44c6-4b60-8260-8efb86e74c4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:24.886358 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4f1443-44c6-4b60-8260-8efb86e74c4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:24.887940 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4f1443-44c6-4b60-8260-8efb86e74c4e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:24.894832 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 799/1594] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:24.895802 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b5161-63eb-452c-9f2b-2a2849e0e3ba admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:24.899703 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 796/1595] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:24.900758 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f1443-44c6-4b60-8260-8efb86e74c4e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:24.910760 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f1443-44c6-4b60-8260-8efb86e74c4e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:24.915317 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb06b0-cc29-49e1-8039-bc2ef9442567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:24.915904 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb06b0-cc29-49e1-8039-bc2ef9442567 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:24.916187 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb06b0-cc29-49e1-8039-bc2ef9442567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:24.916474 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb06b0-cc29-49e1-8039-bc2ef9442567 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:24.917116 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb06b0-cc29-49e1-8039-bc2ef9442567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:24.918693 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb06b0-cc29-49e1-8039-bc2ef9442567 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:24.929392 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 797/1596] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:24.930218 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb06b0-cc29-49e1-8039-bc2ef9442567 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:24.939938 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb06b0-cc29-49e1-8039-bc2ef9442567 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:24.946597 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e57c2-b8b1-44eb-88f6-967c57fd8cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:24.947203 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e57c2-b8b1-44eb-88f6-967c57fd8cf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:24.947502 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e57c2-b8b1-44eb-88f6-967c57fd8cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:24.947964 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e57c2-b8b1-44eb-88f6-967c57fd8cf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:24.948393 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326e57c2-b8b1-44eb-88f6-967c57fd8cf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:24.950015 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326e57c2-b8b1-44eb-88f6-967c57fd8cf7 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:24.961092 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b5161-63eb-452c-9f2b-2a2849e0e3ba admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:24.961743 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 798/1597] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:24.962506 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e57c2-b8b1-44eb-88f6-967c57fd8cf7 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:24.965259 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9580704c-be25-419b-9eb3-dc268a34d77b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:24.965874 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580704c-be25-419b-9eb3-dc268a34d77b service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:24.966159 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580704c-be25-419b-9eb3-dc268a34d77b service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:24.966434 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580704c-be25-419b-9eb3-dc268a34d77b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:24.972480 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e57c2-b8b1-44eb-88f6-967c57fd8cf7 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:24.976881 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9580704c-be25-419b-9eb3-dc268a34d77b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:24.977577 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded4a263-10f7-4553-8d39-7f405a7f7b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:24.978426 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4a263-10f7-4553-8d39-7f405a7f7b54 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:24.978648 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9580704c-be25-419b-9eb3-dc268a34d77b service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:24.978769 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4a263-10f7-4553-8d39-7f405a7f7b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:24.978983 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4a263-10f7-4553-8d39-7f405a7f7b54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:24.979645 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded4a263-10f7-4553-8d39-7f405a7f7b54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:24.981295 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded4a263-10f7-4553-8d39-7f405a7f7b54 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:24.988168 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 800/1598] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:06:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:06:24.989610 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43305a5-7cb2-4e82-8bf1-4005c00999ca None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:24.989897 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43305a5-7cb2-4e82-8bf1-4005c00999ca None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:24.990176 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43305a5-7cb2-4e82-8bf1-4005c00999ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:24.996974 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 799/1599] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:24.996974 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded4a263-10f7-4553-8d39-7f405a7f7b54 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:25.005399 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded4a263-10f7-4553-8d39-7f405a7f7b54 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:25.010945 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1dbcd-da70-4749-9867-ac77489785d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:25.011593 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1dbcd-da70-4749-9867-ac77489785d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:25.011872 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1dbcd-da70-4749-9867-ac77489785d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:25.012143 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1dbcd-da70-4749-9867-ac77489785d5 admin admin] PATH_INFO: `/v3/projects/dcc7379edc954077ab3bf936802fd29f/users/636acaae4bcb43e5bb7a3205e816d0be/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:25.013287 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-b43305a5-7cb2-4e82-8bf1-4005c00999ca None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:25.014587 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b43305a5-7cb2-4e82-8bf1-4005c00999ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:25.025338 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 801/1600] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:25.025338 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43305a5-7cb2-4e82-8bf1-4005c00999ca None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:25.029396 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1dbcd-da70-4749-9867-ac77489785d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcc7379edc954077ab3bf936802fd29f, user_id=636acaae4bcb43e5bb7a3205e816d0be, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:25.030465 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:25.033056 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1dbcd-da70-4749-9867-ac77489785d5 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:25.036250 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43305a5-7cb2-4e82-8bf1-4005c00999ca None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:25.041013 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2605f-1e60-41e9-a7cc-d2692d5853fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:25.041661 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2605f-1e60-41e9-a7cc-d2692d5853fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:25.041969 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2605f-1e60-41e9-a7cc-d2692d5853fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:25.042209 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2605f-1e60-41e9-a7cc-d2692d5853fa admin admin] PATH_INFO: `/v3/projects/3acb006051604f59a1ddcf7a8e7145da/users/a28ff717beef47189f1d005fe7f65a10/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:25.056649 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad2605f-1e60-41e9-a7cc-d2692d5853fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=3acb006051604f59a1ddcf7a8e7145da, user_id=a28ff717beef47189f1d005fe7f65a10, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:25.059357 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad2605f-1e60-41e9-a7cc-d2692d5853fa admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:25.139119 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 802/1601] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:06:25 2022] PUT /identity/v3/projects/3acb006051604f59a1ddcf7a8e7145da/users/a28ff717beef47189f1d005fe7f65a10/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:25.140200 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2605f-1e60-41e9-a7cc-d2692d5853fa admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:25.140919 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 800/1602] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:06:24 2022] PUT /identity/v3/projects/dcc7379edc954077ab3bf936802fd29f/users/636acaae4bcb43e5bb7a3205e816d0be/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:25.142190 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1dbcd-da70-4749-9867-ac77489785d5 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:25.211342 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2605f-1e60-41e9-a7cc-d2692d5853fa admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:25.227122 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1dbcd-da70-4749-9867-ac77489785d5 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:25.232297 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f958c82b-54bd-4c89-a620-1397b82f9f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:25.233074 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958c82b-54bd-4c89-a620-1397b82f9f43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:25.233358 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958c82b-54bd-4c89-a620-1397b82f9f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:25.233632 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958c82b-54bd-4c89-a620-1397b82f9f43 admin admin] PATH_INFO: `/v3/projects/abb894b6991941f48fb5f587ef117110/users/3ca5ca1b4a0a4f62b06a59d456806802/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:25.249663 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f958c82b-54bd-4c89-a620-1397b82f9f43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abb894b6991941f48fb5f587ef117110, user_id=3ca5ca1b4a0a4f62b06a59d456806802, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:25.252394 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f958c82b-54bd-4c89-a620-1397b82f9f43 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:25.279634 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c98b6c-177c-4e27-974f-3c7fbfb61f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:25.280291 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c98b6c-177c-4e27-974f-3c7fbfb61f39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:25.280615 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c98b6c-177c-4e27-974f-3c7fbfb61f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:25.280896 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c98b6c-177c-4e27-974f-3c7fbfb61f39 admin admin] PATH_INFO: `/v3/projects/51dbf47189fc4f3c90eadc4887a831b8/users/9bef8dc1da4c44dbbafcaf3c813b3262/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:25.296586 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c98b6c-177c-4e27-974f-3c7fbfb61f39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51dbf47189fc4f3c90eadc4887a831b8, user_id=9bef8dc1da4c44dbbafcaf3c813b3262, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:25.299320 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c98b6c-177c-4e27-974f-3c7fbfb61f39 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:25.345050 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 801/1603] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:06:25 2022] PUT /identity/v3/projects/abb894b6991941f48fb5f587ef117110/users/3ca5ca1b4a0a4f62b06a59d456806802/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:25.352688 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f958c82b-54bd-4c89-a620-1397b82f9f43 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:25.379288 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 803/1604] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:06:25 2022] PUT /identity/v3/projects/51dbf47189fc4f3c90eadc4887a831b8/users/9bef8dc1da4c44dbbafcaf3c813b3262/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:25.383959 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c98b6c-177c-4e27-974f-3c7fbfb61f39 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:25.388196 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:25.418740 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f958c82b-54bd-4c89-a620-1397b82f9f43 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:25.458756 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c98b6c-177c-4e27-974f-3c7fbfb61f39 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:25.463279 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d3076b-b29a-4a86-ada1-e96a82bdf254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:25.463936 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3076b-b29a-4a86-ada1-e96a82bdf254 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:25.464212 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3076b-b29a-4a86-ada1-e96a82bdf254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:25.464513 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3076b-b29a-4a86-ada1-e96a82bdf254 admin admin] PATH_INFO: `/v3/projects/689f614910a64bc78022bfb4505feba0/users/014ce9fd70c0440ba5973239f9c496a9/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:25.475925 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737eed2-3167-42f6-b776-16df2a05ca36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:25.479594 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737eed2-3167-42f6-b776-16df2a05ca36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:25.479594 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737eed2-3167-42f6-b776-16df2a05ca36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:25.479594 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737eed2-3167-42f6-b776-16df2a05ca36 admin admin] PATH_INFO: `/v3/projects/5a390896075b4d7fb32eb2bfd80f2431/users/b5e4b2720c0645b6b8c20974f28412ea/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:25.479885 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:25.486593 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d3076b-b29a-4a86-ada1-e96a82bdf254 admin admin] RBAC: Authorizing `identity:create_grant(project_id=689f614910a64bc78022bfb4505feba0, user_id=014ce9fd70c0440ba5973239f9c496a9, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:25.489899 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d3076b-b29a-4a86-ada1-e96a82bdf254 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:25.494794 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737eed2-3167-42f6-b776-16df2a05ca36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a390896075b4d7fb32eb2bfd80f2431, user_id=b5e4b2720c0645b6b8c20974f28412ea, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:25.497776 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737eed2-3167-42f6-b776-16df2a05ca36 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:25.614640 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 802/1605] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:06:25 2022] PUT /identity/v3/projects/5a390896075b4d7fb32eb2bfd80f2431/users/b5e4b2720c0645b6b8c20974f28412ea/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:25.626548 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737eed2-3167-42f6-b776-16df2a05ca36 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:25.631027 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 804/1606] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:06:25 2022] PUT /identity/v3/projects/689f614910a64bc78022bfb4505feba0/users/014ce9fd70c0440ba5973239f9c496a9/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:25.632298 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dbe59-607b-44f5-ac6a-2f661f5bb775 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:25.632610 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dbe59-607b-44f5-ac6a-2f661f5bb775 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:25.632878 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dbe59-607b-44f5-ac6a-2f661f5bb775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:25.652535 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-217dbe59-607b-44f5-ac6a-2f661f5bb775 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:25.653754 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-217dbe59-607b-44f5-ac6a-2f661f5bb775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:25.686048 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737eed2-3167-42f6-b776-16df2a05ca36 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:25.713481 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 805/1607] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:25.717620 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc59775-f19c-4ab6-90fb-8c8d0dc3ced0 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:25.717906 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc59775-f19c-4ab6-90fb-8c8d0dc3ced0 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:25.718187 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc59775-f19c-4ab6-90fb-8c8d0dc3ced0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:25.737808 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-ddc59775-f19c-4ab6-90fb-8c8d0dc3ced0 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:25.739029 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ddc59775-f19c-4ab6-90fb-8c8d0dc3ced0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:25.745682 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868cff36-f2c8-4085-b041-51e94dc6bcf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:25.746304 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868cff36-f2c8-4085-b041-51e94dc6bcf8 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:25.746589 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868cff36-f2c8-4085-b041-51e94dc6bcf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:25.746853 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868cff36-f2c8-4085-b041-51e94dc6bcf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:25.747116 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 806/1608] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:25.748350 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291e840-2c5c-42f3-ba50-2ade1c052be4 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:25.748734 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291e840-2c5c-42f3-ba50-2ade1c052be4 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:25.749009 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291e840-2c5c-42f3-ba50-2ade1c052be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:25.759831 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868cff36-f2c8-4085-b041-51e94dc6bcf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:25.761520 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868cff36-f2c8-4085-b041-51e94dc6bcf8 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:25.773079 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 803/1609] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:06:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:06:25.774330 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f6184-b509-4244-aa0f-876e9f036d18 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:25.774609 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f6184-b509-4244-aa0f-876e9f036d18 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:25.774919 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f6184-b509-4244-aa0f-876e9f036d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:25.787181 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-4291e840-2c5c-42f3-ba50-2ade1c052be4 None None] MFA Rules not processed for user `3ca5ca1b4a0a4f62b06a59d456806802`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:25.788694 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-4291e840-2c5c-42f3-ba50-2ade1c052be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:25.795158 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-513f6184-b509-4244-aa0f-876e9f036d18 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:25.796350 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-513f6184-b509-4244-aa0f-876e9f036d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:25.805493 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 804/1610] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:25.806725 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724483b8-19a4-4057-b71b-a2f30cd1b80d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:25.807010 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724483b8-19a4-4057-b71b-a2f30cd1b80d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:25.807282 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724483b8-19a4-4057-b71b-a2f30cd1b80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:25.839410 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-724483b8-19a4-4057-b71b-a2f30cd1b80d None None] MFA Rules not processed for user `b5e4b2720c0645b6b8c20974f28412ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:25.840722 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-724483b8-19a4-4057-b71b-a2f30cd1b80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:25.844850 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:25.884097 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 807/1611] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:25.885403 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73505b8b-86be-4ecb-a738-9120ab361240 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:25.885744 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73505b8b-86be-4ecb-a738-9120ab361240 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:25.886020 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73505b8b-86be-4ecb-a738-9120ab361240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:25.914438 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 805/1612] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:25.915668 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724483b8-19a4-4057-b71b-a2f30cd1b80d None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:25.927451 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-73505b8b-86be-4ecb-a738-9120ab361240 None None] MFA Rules not processed for user `014ce9fd70c0440ba5973239f9c496a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:25.928749 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-73505b8b-86be-4ecb-a738-9120ab361240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:25.929551 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724483b8-19a4-4057-b71b-a2f30cd1b80d None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:25.947132 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bf8e29-4bd0-4d19-b256-eba2af50c632 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:25.947784 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf8e29-4bd0-4d19-b256-eba2af50c632 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:25.948070 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf8e29-4bd0-4d19-b256-eba2af50c632 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:25.948347 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf8e29-4bd0-4d19-b256-eba2af50c632 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:25.966858 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bf8e29-4bd0-4d19-b256-eba2af50c632 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:25.968766 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bf8e29-4bd0-4d19-b256-eba2af50c632 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:26.000985 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 806/1613] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:06:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:06:26.004199 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bf8e29-4bd0-4d19-b256-eba2af50c632 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:26.029009 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bf8e29-4bd0-4d19-b256-eba2af50c632 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:26.030388 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 808/1614] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:26.031733 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73505b8b-86be-4ecb-a738-9120ab361240 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:26.040136 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:26.040839 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab3b32b-0a73-4839-a1b7-f4cda590d061 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:26.041474 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3b32b-0a73-4839-a1b7-f4cda590d061 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:26.042451 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3b32b-0a73-4839-a1b7-f4cda590d061 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:26.042797 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3b32b-0a73-4839-a1b7-f4cda590d061 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:26.045795 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73505b8b-86be-4ecb-a738-9120ab361240 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:26.051374 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2b99f-5d8a-4c1b-9955-5bc3329581f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:26.052738 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2b99f-5d8a-4c1b-9955-5bc3329581f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:26.053955 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2b99f-5d8a-4c1b-9955-5bc3329581f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:26.054446 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2b99f-5d8a-4c1b-9955-5bc3329581f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:26.056054 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab3b32b-0a73-4839-a1b7-f4cda590d061 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:26.057812 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab3b32b-0a73-4839-a1b7-f4cda590d061 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:26.067426 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f2b99f-5d8a-4c1b-9955-5bc3329581f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:26.069295 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f2b99f-5d8a-4c1b-9955-5bc3329581f5 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:26.069436 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 807/1615] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:06:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:06:26.082447 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab3b32b-0a73-4839-a1b7-f4cda590d061 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:26.082865 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 809/1616] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:06:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:06:26.084526 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087432d-465a-418c-bb81-7e50768f7f6a None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:26.085164 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087432d-465a-418c-bb81-7e50768f7f6a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:26.085913 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087432d-465a-418c-bb81-7e50768f7f6a None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:26.087730 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 810/1617] 10.222.0.27 () {54 vars in 962 bytes} [Sun Sep 4 01:06:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:06:26.088636 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087432d-465a-418c-bb81-7e50768f7f6a None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:26.108775 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087432d-465a-418c-bb81-7e50768f7f6a None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:26.111895 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab3b32b-0a73-4839-a1b7-f4cda590d061 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:26.115894 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e615e5-7645-422e-9b7e-cf415741fb6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:26.117010 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e615e5-7645-422e-9b7e-cf415741fb6f admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:26.117542 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e615e5-7645-422e-9b7e-cf415741fb6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:26.118808 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e615e5-7645-422e-9b7e-cf415741fb6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:26.119624 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e615e5-7645-422e-9b7e-cf415741fb6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:26.122465 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e615e5-7645-422e-9b7e-cf415741fb6f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:26.125344 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f3abb5-f887-4dd7-869c-a1a7d8661085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:26.125970 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3abb5-f887-4dd7-869c-a1a7d8661085 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:26.126247 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3abb5-f887-4dd7-869c-a1a7d8661085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:26.126519 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3abb5-f887-4dd7-869c-a1a7d8661085 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:26.128938 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f3abb5-f887-4dd7-869c-a1a7d8661085 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:26.131856 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f3abb5-f887-4dd7-869c-a1a7d8661085 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:26.241529 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 811/1618] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:06:26 2022] POST /identity/v3/projects => generated 365 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:26.243245 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219709e3-ace3-4ac4-aa85-b450916ee38a None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:26.243660 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219709e3-ace3-4ac4-aa85-b450916ee38a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:26.244203 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219709e3-ace3-4ac4-aa85-b450916ee38a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:26.275599 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 808/1619] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:06:26 2022] POST /identity/v3/projects => generated 385 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:26.277265 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f3abb5-f887-4dd7-869c-a1a7d8661085 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:26.330381 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-219709e3-ace3-4ac4-aa85-b450916ee38a None None] MFA Rules not processed for user `68bc00bda5194cd8a7928f8c4df4ccd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:26.333135 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-219709e3-ace3-4ac4-aa85-b450916ee38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:26.415111 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f3abb5-f887-4dd7-869c-a1a7d8661085 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:26.420816 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b9318-4798-4d56-ae9b-bbe7bf9c5a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:26.421378 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b9318-4798-4d56-ae9b-bbe7bf9c5a5b admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:26.421818 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b9318-4798-4d56-ae9b-bbe7bf9c5a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:26.422093 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b9318-4798-4d56-ae9b-bbe7bf9c5a5b admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:26.422853 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3b9318-4798-4d56-ae9b-bbe7bf9c5a5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:26.425881 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3b9318-4798-4d56-ae9b-bbe7bf9c5a5b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:26.448318 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 812/1620] 10.222.0.27 () {58 vars in 1059 bytes} [Sun Sep 4 01:06:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:26.449571 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219709e3-ace3-4ac4-aa85-b450916ee38a None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:26.462784 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219709e3-ace3-4ac4-aa85-b450916ee38a None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:26.468420 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e73ea-cc64-4d8d-8726-320d89cf8c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:26.469384 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e73ea-cc64-4d8d-8726-320d89cf8c7a admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:26.469522 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e73ea-cc64-4d8d-8726-320d89cf8c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:26.469823 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e73ea-cc64-4d8d-8726-320d89cf8c7a admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:26.470584 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e73ea-cc64-4d8d-8726-320d89cf8c7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:26.472921 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e73ea-cc64-4d8d-8726-320d89cf8c7a admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:26.587427 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 809/1621] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:06:26 2022] POST /identity/v3/users => generated 289 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:26.588297 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b9318-4798-4d56-ae9b-bbe7bf9c5a5b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:26.601058 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b9318-4798-4d56-ae9b-bbe7bf9c5a5b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:26.606908 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d5c921-64f5-4bb9-a86f-92375c7fd7ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:26.607933 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5c921-64f5-4bb9-a86f-92375c7fd7ce service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:26.613345 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5c921-64f5-4bb9-a86f-92375c7fd7ce service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:26.613345 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5c921-64f5-4bb9-a86f-92375c7fd7ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:26.620041 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 813/1622] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:06:26 2022] POST /identity/v3/users => generated 299 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:26.620041 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e73ea-cc64-4d8d-8726-320d89cf8c7a admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:26.632393 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e73ea-cc64-4d8d-8726-320d89cf8c7a admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:26.638052 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83731fef-01b1-447e-a8e6-8a5e51bbf56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:26.638697 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83731fef-01b1-447e-a8e6-8a5e51bbf56f admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:26.639074 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83731fef-01b1-447e-a8e6-8a5e51bbf56f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:26.639466 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83731fef-01b1-447e-a8e6-8a5e51bbf56f admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:26.640666 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83731fef-01b1-447e-a8e6-8a5e51bbf56f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:26.642736 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83731fef-01b1-447e-a8e6-8a5e51bbf56f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:26.669365 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 814/1623] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:26.670583 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83731fef-01b1-447e-a8e6-8a5e51bbf56f admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:26.684879 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:26.689048 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83731fef-01b1-447e-a8e6-8a5e51bbf56f admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:26.696070 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae070cd-eae3-47ba-a736-cf75cee67179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:26.697074 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae070cd-eae3-47ba-a736-cf75cee67179 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:26.697523 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae070cd-eae3-47ba-a736-cf75cee67179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:26.697956 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae070cd-eae3-47ba-a736-cf75cee67179 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:26.698801 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae070cd-eae3-47ba-a736-cf75cee67179 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:26.700877 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae070cd-eae3-47ba-a736-cf75cee67179 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:26.701954 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d5c921-64f5-4bb9-a86f-92375c7fd7ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:26.704429 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d5c921-64f5-4bb9-a86f-92375c7fd7ce service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:26.716658 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 815/1624] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:26.718347 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae070cd-eae3-47ba-a736-cf75cee67179 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:26.729091 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 810/1625] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:06:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:06:26.731284 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d5c921-64f5-4bb9-a86f-92375c7fd7ce service nova] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:26.731860 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae070cd-eae3-47ba-a736-cf75cee67179 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:26.738301 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a52a8e-4780-4711-a8b0-808dc27008cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:26.739327 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a52a8e-4780-4711-a8b0-808dc27008cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:26.739910 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a52a8e-4780-4711-a8b0-808dc27008cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:26.740546 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a52a8e-4780-4711-a8b0-808dc27008cf admin admin] PATH_INFO: `/v3/projects/73ecaffec16d4a259949066dcd42b36c/users/7cf4744fca984e37a2731b29b5d65227/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:26.757736 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d5c921-64f5-4bb9-a86f-92375c7fd7ce service nova] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:26.760972 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a52a8e-4780-4711-a8b0-808dc27008cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=73ecaffec16d4a259949066dcd42b36c, user_id=7cf4744fca984e37a2731b29b5d65227, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:26.763494 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453a802-5ba5-4e30-b737-f62ff593d050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:26.765311 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453a802-5ba5-4e30-b737-f62ff593d050 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:26.765448 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a52a8e-4780-4711-a8b0-808dc27008cf admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:26.765542 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453a802-5ba5-4e30-b737-f62ff593d050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:26.765632 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453a802-5ba5-4e30-b737-f62ff593d050 admin admin] PATH_INFO: `/v3/projects/ac7da325d9624a4ea0fcce8620bc9c7a/users/a349ef8082da4d789b63c94c8e410593/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:26.783694 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453a802-5ba5-4e30-b737-f62ff593d050 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac7da325d9624a4ea0fcce8620bc9c7a, user_id=a349ef8082da4d789b63c94c8e410593, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:26.786932 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453a802-5ba5-4e30-b737-f62ff593d050 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:26.850002 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 816/1626] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:06:26 2022] PUT /identity/v3/projects/73ecaffec16d4a259949066dcd42b36c/users/7cf4744fca984e37a2731b29b5d65227/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:26.851089 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a52a8e-4780-4711-a8b0-808dc27008cf admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:26.907518 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 811/1627] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:06:26 2022] PUT /identity/v3/projects/ac7da325d9624a4ea0fcce8620bc9c7a/users/a349ef8082da4d789b63c94c8e410593/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:26.908665 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453a802-5ba5-4e30-b737-f62ff593d050 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:26.946399 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:26.991307 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a52a8e-4780-4711-a8b0-808dc27008cf admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:27.004525 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89dc53-4576-421f-8722-93a49d79fa1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:27.004908 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89dc53-4576-421f-8722-93a49d79fa1a admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:27.005186 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89dc53-4576-421f-8722-93a49d79fa1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:27.005458 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89dc53-4576-421f-8722-93a49d79fa1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:27.006126 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453a802-5ba5-4e30-b737-f62ff593d050 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:27.006258 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a89dc53-4576-421f-8722-93a49d79fa1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:27.007720 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a89dc53-4576-421f-8722-93a49d79fa1a admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:27.012105 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c5c4cf-c71e-4777-83a2-e05daf88dae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:27.012997 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5c4cf-c71e-4777-83a2-e05daf88dae3 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:27.013271 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5c4cf-c71e-4777-83a2-e05daf88dae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:27.013538 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5c4cf-c71e-4777-83a2-e05daf88dae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:27.014177 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c5c4cf-c71e-4777-83a2-e05daf88dae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:27.017010 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c5c4cf-c71e-4777-83a2-e05daf88dae3 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:27.032680 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 817/1628] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:27.034140 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 812/1629] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:27.034915 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b8f94-92ed-474a-a6d1-85ee092522b1 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:27.035562 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b8f94-92ed-474a-a6d1-85ee092522b1 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:27.035991 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b8f94-92ed-474a-a6d1-85ee092522b1 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:27.039597 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c5c4cf-c71e-4777-83a2-e05daf88dae3 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:27.044663 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 818/1630] 10.222.0.27 () {54 vars in 912 bytes} [Sun Sep 4 01:06:27 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:06:27.045799 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92b8f94-92ed-474a-a6d1-85ee092522b1 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:27.052951 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c5c4cf-c71e-4777-83a2-e05daf88dae3 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:27.061799 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a5cb9-4116-4274-8509-223dc0af9faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:27.061799 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a5cb9-4116-4274-8509-223dc0af9faa admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:27.061799 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a5cb9-4116-4274-8509-223dc0af9faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:27.061799 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a5cb9-4116-4274-8509-223dc0af9faa admin admin] PATH_INFO: `/v3/projects/73ecaffec16d4a259949066dcd42b36c/users/7cf4744fca984e37a2731b29b5d65227/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:27.063513 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a5cb9-4116-4274-8509-223dc0af9faa admin admin] RBAC: Authorizing `identity:create_grant(project_id=73ecaffec16d4a259949066dcd42b36c, user_id=7cf4744fca984e37a2731b29b5d65227, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:27.064263 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92b8f94-92ed-474a-a6d1-85ee092522b1 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:27.070428 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0913449-088c-4f19-b217-ccd7a954975b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:27.071114 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0913449-088c-4f19-b217-ccd7a954975b admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:27.071401 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0913449-088c-4f19-b217-ccd7a954975b admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:27.071690 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0913449-088c-4f19-b217-ccd7a954975b admin admin] PATH_INFO: `/v3/projects/ac7da325d9624a4ea0fcce8620bc9c7a/users/a349ef8082da4d789b63c94c8e410593/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:27.074457 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0913449-088c-4f19-b217-ccd7a954975b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac7da325d9624a4ea0fcce8620bc9c7a, user_id=a349ef8082da4d789b63c94c8e410593, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:27.075188 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a5cb9-4116-4274-8509-223dc0af9faa admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:27.079042 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0913449-088c-4f19-b217-ccd7a954975b admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:27.104786 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 819/1631] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:06:27 2022] PUT /identity/v3/projects/ac7da325d9624a4ea0fcce8620bc9c7a/users/a349ef8082da4d789b63c94c8e410593/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:27.106697 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602c5a6-eb29-4745-bea3-ce6f3a186bd5 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:27.106838 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 813/1632] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:06:27 2022] PUT /identity/v3/projects/73ecaffec16d4a259949066dcd42b36c/users/7cf4744fca984e37a2731b29b5d65227/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:27.108066 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602c5a6-eb29-4745-bea3-ce6f3a186bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:27.108255 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41e247-79b1-4277-b12c-1508c3ce2734 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:27.108693 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41e247-79b1-4277-b12c-1508c3ce2734 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:27.109036 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41e247-79b1-4277-b12c-1508c3ce2734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:27.109288 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602c5a6-eb29-4745-bea3-ce6f3a186bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:27.140409 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-c602c5a6-eb29-4745-bea3-ce6f3a186bd5 None None] MFA Rules not processed for user `68bc00bda5194cd8a7928f8c4df4ccd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:27.142497 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c602c5a6-eb29-4745-bea3-ce6f3a186bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:27.169934 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-0e41e247-79b1-4277-b12c-1508c3ce2734 None None] MFA Rules not processed for user `a349ef8082da4d789b63c94c8e410593`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:27.171294 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0e41e247-79b1-4277-b12c-1508c3ce2734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:27.256109 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 820/1633] 10.222.0.27 () {58 vars in 1009 bytes} [Sun Sep 4 01:06:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:27.257725 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee992f-5976-49b6-a703-e4a68ca4bc3d None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:27.258094 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee992f-5976-49b6-a703-e4a68ca4bc3d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:27.258380 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee992f-5976-49b6-a703-e4a68ca4bc3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:27.295791 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-3aee992f-5976-49b6-a703-e4a68ca4bc3d None None] MFA Rules not processed for user `7cf4744fca984e37a2731b29b5d65227`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:27.297641 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3aee992f-5976-49b6-a703-e4a68ca4bc3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:27.323529 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 814/1634] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:27 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:27.324662 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41e247-79b1-4277-b12c-1508c3ce2734 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:27.422377 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41e247-79b1-4277-b12c-1508c3ce2734 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:27.428724 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ceb4c-acce-4c86-a0ab-0f40aff94715 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:27.429018 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 821/1635] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:27 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:27.430504 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ceb4c-acce-4c86-a0ab-0f40aff94715 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:27.430895 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ceb4c-acce-4c86-a0ab-0f40aff94715 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:27.431273 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ceb4c-acce-4c86-a0ab-0f40aff94715 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:27.435517 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee992f-5976-49b6-a703-e4a68ca4bc3d None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:27.445828 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee992f-5976-49b6-a703-e4a68ca4bc3d None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:27.449610 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76223be-2225-48e8-88f5-cb6feab194ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:27.450238 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76223be-2225-48e8-88f5-cb6feab194ce service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:27.458626 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76223be-2225-48e8-88f5-cb6feab194ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:27.459090 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76223be-2225-48e8-88f5-cb6feab194ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:27.461012 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ceb4c-acce-4c86-a0ab-0f40aff94715 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:27.462476 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ceb4c-acce-4c86-a0ab-0f40aff94715 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:27.470618 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76223be-2225-48e8-88f5-cb6feab194ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:27.473100 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76223be-2225-48e8-88f5-cb6feab194ce service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:27.486956 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 822/1637] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:06:27 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:06:27.488557 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 815/1637] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:06:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:06:27.522597 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:27.532092 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76223be-2225-48e8-88f5-cb6feab194ce service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:27.538583 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ceb4c-acce-4c86-a0ab-0f40aff94715 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:27.548378 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ceb4c-acce-4c86-a0ab-0f40aff94715 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:27.561440 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa3881-d2d4-46e8-947a-fdc8378f71d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:27.562196 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa3881-d2d4-46e8-947a-fdc8378f71d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:27.564824 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa3881-d2d4-46e8-947a-fdc8378f71d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:27.565227 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa3881-d2d4-46e8-947a-fdc8378f71d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:27.590940 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa3881-d2d4-46e8-947a-fdc8378f71d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:27.592586 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa3881-d2d4-46e8-947a-fdc8378f71d1 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:27.609196 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76223be-2225-48e8-88f5-cb6feab194ce service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:27.615825 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c04456-3b04-413c-863f-d21ebb19ed05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:27.616514 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c04456-3b04-413c-863f-d21ebb19ed05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:27.616808 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c04456-3b04-413c-863f-d21ebb19ed05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:27.617092 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c04456-3b04-413c-863f-d21ebb19ed05 admin admin] PATH_INFO: `/v3/users/3ca5ca1b4a0a4f62b06a59d456806802` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:27.618435 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c04456-3b04-413c-863f-d21ebb19ed05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ca5ca1b4a0a4f62b06a59d456806802)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:27.619847 dv-cinder-master-3-smb devstack@keystone.service[81630]: WARNING py.warnings [None req-42c04456-3b04-413c-863f-d21ebb19ed05 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 04 01:06:27.619847 dv-cinder-master-3-smb devstack@keystone.service[81630]: warnings.warn(msg) Sep 04 01:06:27.619847 dv-cinder-master-3-smb devstack@keystone.service[81630]: Sep 04 01:06:27.621503 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c04456-3b04-413c-863f-d21ebb19ed05 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:27.630779 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 816/1638] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:06:27 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 01:06:27.886847 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:27.992004 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-42c04456-3b04-413c-863f-d21ebb19ed05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ca5ca1b4a0a4f62b06a59d456806802'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:27.993785 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-42c04456-3b04-413c-863f-d21ebb19ed05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ca5ca1b4a0a4f62b06a59d456806802'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:27.994535 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa3881-d2d4-46e8-947a-fdc8378f71d1 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:28.007252 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa3881-d2d4-46e8-947a-fdc8378f71d1 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:28.017338 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41f1b1-6d14-4229-b69d-c296a5f4cd8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:28.018056 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41f1b1-6d14-4229-b69d-c296a5f4cd8c service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:28.018416 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41f1b1-6d14-4229-b69d-c296a5f4cd8c service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:28.018828 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41f1b1-6d14-4229-b69d-c296a5f4cd8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:28.020840 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7c41f1b1-6d14-4229-b69d-c296a5f4cd8c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:28.066591 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-42c04456-3b04-413c-863f-d21ebb19ed05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ca5ca1b4a0a4f62b06a59d456806802'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:28.069582 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-42c04456-3b04-413c-863f-d21ebb19ed05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ca5ca1b4a0a4f62b06a59d456806802'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:28.125785 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-42c04456-3b04-413c-863f-d21ebb19ed05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ca5ca1b4a0a4f62b06a59d456806802'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:28.159685 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 823/1639] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:06:27 2022] DELETE /identity/v3/users/3ca5ca1b4a0a4f62b06a59d456806802 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:28.163824 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c1f8f-0d0d-454f-b3a5-c373a720e28a None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:28.165544 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c1f8f-0d0d-454f-b3a5-c373a720e28a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:28.165920 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c1f8f-0d0d-454f-b3a5-c373a720e28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:28.177181 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c41f1b1-6d14-4229-b69d-c296a5f4cd8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:28.182265 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c41f1b1-6d14-4229-b69d-c296a5f4cd8c service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:28.218941 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 817/1640] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:06:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:06:28.224294 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-6d7c1f8f-0d0d-454f-b3a5-c373a720e28a None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:28.225567 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-6d7c1f8f-0d0d-454f-b3a5-c373a720e28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:28.594199 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 824/1641] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:06:28.594618 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41f1b1-6d14-4229-b69d-c296a5f4cd8c service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:28.594618 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7c41f1b1-6d14-4229-b69d-c296a5f4cd8c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:28.594618 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41f1b1-6d14-4229-b69d-c296a5f4cd8c service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:28.608307 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e982f0f-6ed7-4f84-815f-23ded612a9b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:28.609051 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e982f0f-6ed7-4f84-815f-23ded612a9b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:28.609430 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e982f0f-6ed7-4f84-815f-23ded612a9b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:28.609777 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e982f0f-6ed7-4f84-815f-23ded612a9b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:28.623195 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e982f0f-6ed7-4f84-815f-23ded612a9b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:28.625381 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e982f0f-6ed7-4f84-815f-23ded612a9b7 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:28.639554 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 818/1642] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:06:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:06:28.755663 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873db7d2-7ec9-46dc-aab0-9284d8a4a971 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:28.756709 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873db7d2-7ec9-46dc-aab0-9284d8a4a971 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:28.757064 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873db7d2-7ec9-46dc-aab0-9284d8a4a971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:28.798770 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-873db7d2-7ec9-46dc-aab0-9284d8a4a971 None None] MFA Rules not processed for user `636acaae4bcb43e5bb7a3205e816d0be`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:28.800777 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-873db7d2-7ec9-46dc-aab0-9284d8a4a971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:28.922099 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e982f0f-6ed7-4f84-815f-23ded612a9b7 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:28.923576 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4e982f0f-6ed7-4f84-815f-23ded612a9b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:28.926336 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 825/1643] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:28 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:28.944241 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e982f0f-6ed7-4f84-815f-23ded612a9b7 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:28.949990 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3648966a-1e9a-48fe-9a0e-b516084b22d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:28.950613 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648966a-1e9a-48fe-9a0e-b516084b22d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:28.950893 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648966a-1e9a-48fe-9a0e-b516084b22d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:28.951164 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648966a-1e9a-48fe-9a0e-b516084b22d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:28.951870 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3648966a-1e9a-48fe-9a0e-b516084b22d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:28.955719 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3648966a-1e9a-48fe-9a0e-b516084b22d7 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:28.979584 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:29.124583 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873db7d2-7ec9-46dc-aab0-9284d8a4a971 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:29.132930 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 819/1644] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:06:28 2022] POST /identity/v3/projects => generated 405 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:29.143872 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873db7d2-7ec9-46dc-aab0-9284d8a4a971 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:29.291678 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa91ca4c-158d-413a-b8af-11a2b6bcb0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:29.294895 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91ca4c-158d-413a-b8af-11a2b6bcb0df admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:29.310460 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91ca4c-158d-413a-b8af-11a2b6bcb0df admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:29.310460 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91ca4c-158d-413a-b8af-11a2b6bcb0df admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:29.310460 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa91ca4c-158d-413a-b8af-11a2b6bcb0df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:29.310460 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa91ca4c-158d-413a-b8af-11a2b6bcb0df admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:29.382881 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199c262-defd-4671-a893-c7d9cdbb3037 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:29.383961 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199c262-defd-4671-a893-c7d9cdbb3037 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:29.384263 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199c262-defd-4671-a893-c7d9cdbb3037 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:29.385895 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 820/1645] 10.222.0.27 () {54 vars in 962 bytes} [Sun Sep 4 01:06:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 01:06:29.394922 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a923e6-d8b0-49a8-b9a2-f3e09fa2e449 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:29.399286 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a923e6-d8b0-49a8-b9a2-f3e09fa2e449 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:29.399286 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a923e6-d8b0-49a8-b9a2-f3e09fa2e449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:29.421006 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:29.432483 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-53a923e6-d8b0-49a8-b9a2-f3e09fa2e449 None None] MFA Rules not processed for user `68bc00bda5194cd8a7928f8c4df4ccd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:29.434473 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-53a923e6-d8b0-49a8-b9a2-f3e09fa2e449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:29.457238 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 826/1646] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:06:29 2022] POST /identity/v3/users => generated 309 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:29.457238 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa91ca4c-158d-413a-b8af-11a2b6bcb0df admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:29.483099 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa91ca4c-158d-413a-b8af-11a2b6bcb0df admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:29.491335 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d239a-0545-4e27-a752-6f334a6b5625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:29.492285 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d239a-0545-4e27-a752-6f334a6b5625 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:29.492813 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d239a-0545-4e27-a752-6f334a6b5625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:29.493393 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d239a-0545-4e27-a752-6f334a6b5625 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:29.494244 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d239a-0545-4e27-a752-6f334a6b5625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:29.497260 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d239a-0545-4e27-a752-6f334a6b5625 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:29.512267 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 827/1647] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:29.518480 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d239a-0545-4e27-a752-6f334a6b5625 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:29.533305 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d239a-0545-4e27-a752-6f334a6b5625 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:29.543638 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096a92b-76d1-4716-9378-095116325d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:29.545896 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096a92b-76d1-4716-9378-095116325d64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:29.546553 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096a92b-76d1-4716-9378-095116325d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:29.546943 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096a92b-76d1-4716-9378-095116325d64 admin admin] PATH_INFO: `/v3/projects/a7f4995ce2c443a6924b53184dc2df32/users/0cabbafc1e8b4263b740017ead3a1f8d/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:29.572881 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4096a92b-76d1-4716-9378-095116325d64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7f4995ce2c443a6924b53184dc2df32, user_id=0cabbafc1e8b4263b740017ead3a1f8d, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:29.577040 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4096a92b-76d1-4716-9378-095116325d64 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:29.579022 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 821/1648] 10.222.0.27 () {58 vars in 1059 bytes} [Sun Sep 4 01:06:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:06:29.585431 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde64a2-2c34-4d2d-95d2-6e107068a10a None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:29.585716 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde64a2-2c34-4d2d-95d2-6e107068a10a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:29.585989 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde64a2-2c34-4d2d-95d2-6e107068a10a None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:29.587300 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 822/1649] 10.222.0.27 () {54 vars in 912 bytes} [Sun Sep 4 01:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 01:06:29.592763 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f4451-004e-44ca-b9df-8c7ffe7c574c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:29.593046 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f4451-004e-44ca-b9df-8c7ffe7c574c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:29.593369 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f4451-004e-44ca-b9df-8c7ffe7c574c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:29.617792 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-685f4451-004e-44ca-b9df-8c7ffe7c574c None None] MFA Rules not processed for user `68bc00bda5194cd8a7928f8c4df4ccd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:29.618990 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-685f4451-004e-44ca-b9df-8c7ffe7c574c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:29.630024 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 823/1650] 10.222.0.27 () {58 vars in 1009 bytes} [Sun Sep 4 01:06:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:29.645870 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f4451-004e-44ca-b9df-8c7ffe7c574c None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:29.648146 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-685f4451-004e-44ca-b9df-8c7ffe7c574c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:29.661036 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 828/1651] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:06:29 2022] PUT /identity/v3/projects/a7f4995ce2c443a6924b53184dc2df32/users/0cabbafc1e8b4263b740017ead3a1f8d/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:06:29.665593 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096a92b-76d1-4716-9378-095116325d64 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:29.732671 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f4451-004e-44ca-b9df-8c7ffe7c574c None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:29.771376 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096a92b-76d1-4716-9378-095116325d64 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:29.777934 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa02af-b0a2-4f2a-9d29-99aa80b520cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:29.778560 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa02af-b0a2-4f2a-9d29-99aa80b520cf admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:29.778870 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa02af-b0a2-4f2a-9d29-99aa80b520cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:29.779671 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa02af-b0a2-4f2a-9d29-99aa80b520cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:29.781366 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa02af-b0a2-4f2a-9d29-99aa80b520cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:29.783281 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa02af-b0a2-4f2a-9d29-99aa80b520cf admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:29.802762 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 829/1652] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:29.804803 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad107bd-d54b-49af-bab9-59a0c70747c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:29.805709 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad107bd-d54b-49af-bab9-59a0c70747c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:29.806117 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad107bd-d54b-49af-bab9-59a0c70747c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:29.806515 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad107bd-d54b-49af-bab9-59a0c70747c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:29.806788 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa02af-b0a2-4f2a-9d29-99aa80b520cf admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:29.820487 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa02af-b0a2-4f2a-9d29-99aa80b520cf admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:29.825561 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929667d1-3bc6-48f5-bcb1-23a859c7e0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:29.826240 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929667d1-3bc6-48f5-bcb1-23a859c7e0d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:29.826524 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929667d1-3bc6-48f5-bcb1-23a859c7e0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:29.829495 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929667d1-3bc6-48f5-bcb1-23a859c7e0d4 admin admin] PATH_INFO: `/v3/projects/a7f4995ce2c443a6924b53184dc2df32/users/0cabbafc1e8b4263b740017ead3a1f8d/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:29.829495 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929667d1-3bc6-48f5-bcb1-23a859c7e0d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7f4995ce2c443a6924b53184dc2df32, user_id=0cabbafc1e8b4263b740017ead3a1f8d, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:29.831780 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929667d1-3bc6-48f5-bcb1-23a859c7e0d4 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:29.848009 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 830/1653] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:06:29 2022] PUT /identity/v3/projects/a7f4995ce2c443a6924b53184dc2df32/users/0cabbafc1e8b4263b740017ead3a1f8d/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:29.869198 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:29.901197 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad107bd-d54b-49af-bab9-59a0c70747c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:29.902755 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad107bd-d54b-49af-bab9-59a0c70747c3 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:29.926019 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 824/1654] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:06:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:06:30.110809 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929667d1-3bc6-48f5-bcb1-23a859c7e0d4 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:30.113380 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-929667d1-3bc6-48f5-bcb1-23a859c7e0d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:30.307039 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929667d1-3bc6-48f5-bcb1-23a859c7e0d4 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:30.318622 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c7335c-27c9-451d-99fd-1f2507b9d16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:30.321280 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7335c-27c9-451d-99fd-1f2507b9d16c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:30.321280 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7335c-27c9-451d-99fd-1f2507b9d16c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:30.321280 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7335c-27c9-451d-99fd-1f2507b9d16c admin admin] PATH_INFO: `/v3/projects/abb894b6991941f48fb5f587ef117110` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:30.321280 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c7335c-27c9-451d-99fd-1f2507b9d16c admin admin] RBAC: Authorizing `identity:delete_project(project_id=abb894b6991941f48fb5f587ef117110)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:30.333841 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c7335c-27c9-451d-99fd-1f2507b9d16c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:30.488136 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b8c7335c-27c9-451d-99fd-1f2507b9d16c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abb894b6991941f48fb5f587ef117110', 'request_id': 'req-b8c7335c-27c9-451d-99fd-1f2507b9d16c'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:30.572884 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b8c7335c-27c9-451d-99fd-1f2507b9d16c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abb894b6991941f48fb5f587ef117110', 'request_id': 'req-b8c7335c-27c9-451d-99fd-1f2507b9d16c'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:30.583289 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b8c7335c-27c9-451d-99fd-1f2507b9d16c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abb894b6991941f48fb5f587ef117110', 'request_id': 'req-b8c7335c-27c9-451d-99fd-1f2507b9d16c'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:30.620014 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b8c7335c-27c9-451d-99fd-1f2507b9d16c admin admin] The token cache is being invalidate because project abb894b6991941f48fb5f587ef117110 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:06:30.620597 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b8c7335c-27c9-451d-99fd-1f2507b9d16c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:30.623760 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b8c7335c-27c9-451d-99fd-1f2507b9d16c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:30.626979 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 831/1655] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:06:30 2022] DELETE /identity/v3/projects/abb894b6991941f48fb5f587ef117110 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:30.632330 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad107bd-d54b-49af-bab9-59a0c70747c3 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:30.651027 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad107bd-d54b-49af-bab9-59a0c70747c3 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:30.815206 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7a7ec-923b-48ca-bd26-8e125efa33d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:30.816665 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7a7ec-923b-48ca-bd26-8e125efa33d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:30.817310 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7a7ec-923b-48ca-bd26-8e125efa33d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:30.817439 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7a7ec-923b-48ca-bd26-8e125efa33d8 admin admin] PATH_INFO: `/v3/users/a349ef8082da4d789b63c94c8e410593` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:30.819078 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad7a7ec-923b-48ca-bd26-8e125efa33d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a349ef8082da4d789b63c94c8e410593)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:30.821165 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad7a7ec-923b-48ca-bd26-8e125efa33d8 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:31.093683 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8ad7a7ec-923b-48ca-bd26-8e125efa33d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a349ef8082da4d789b63c94c8e410593'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:31.095873 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8ad7a7ec-923b-48ca-bd26-8e125efa33d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a349ef8082da4d789b63c94c8e410593'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:31.129255 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8ad7a7ec-923b-48ca-bd26-8e125efa33d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a349ef8082da4d789b63c94c8e410593'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:31.209309 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8ad7a7ec-923b-48ca-bd26-8e125efa33d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a349ef8082da4d789b63c94c8e410593'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:31.210060 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8ad7a7ec-923b-48ca-bd26-8e125efa33d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a349ef8082da4d789b63c94c8e410593'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:31.255996 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 825/1656] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:06:30 2022] DELETE /identity/v3/users/a349ef8082da4d789b63c94c8e410593 => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:31.812196 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:33.018452 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb3032-55f7-48d0-9849-3156d678e31a None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:33.019980 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb3032-55f7-48d0-9849-3156d678e31a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:33.020729 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb3032-55f7-48d0-9849-3156d678e31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:33.073360 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-76eb3032-55f7-48d0-9849-3156d678e31a None None] MFA Rules not processed for user `0cabbafc1e8b4263b740017ead3a1f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:33.075997 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-76eb3032-55f7-48d0-9849-3156d678e31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:33.209262 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 832/1657] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:33 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:33.770495 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7a7ec-923b-48ca-bd26-8e125efa33d8 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:33.777732 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8ad7a7ec-923b-48ca-bd26-8e125efa33d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:33.946325 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:33.958199 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7a7ec-923b-48ca-bd26-8e125efa33d8 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:33.989471 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccfbe55-075a-40b7-8e32-5e4dfcc786b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:33.989471 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccfbe55-075a-40b7-8e32-5e4dfcc786b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:33.991567 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccfbe55-075a-40b7-8e32-5e4dfcc786b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:33.991769 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccfbe55-075a-40b7-8e32-5e4dfcc786b9 admin admin] PATH_INFO: `/v3/projects/ac7da325d9624a4ea0fcce8620bc9c7a` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:34.001022 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccfbe55-075a-40b7-8e32-5e4dfcc786b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac7da325d9624a4ea0fcce8620bc9c7a)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:34.002091 dv-cinder-master-3-smb devstack@keystone.service[81629]: WARNING py.warnings [None req-5ccfbe55-075a-40b7-8e32-5e4dfcc786b9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 04 01:06:34.002091 dv-cinder-master-3-smb devstack@keystone.service[81629]: warnings.warn(msg) Sep 04 01:06:34.002091 dv-cinder-master-3-smb devstack@keystone.service[81629]: Sep 04 01:06:34.003711 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccfbe55-075a-40b7-8e32-5e4dfcc786b9 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:34.242834 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5ccfbe55-075a-40b7-8e32-5e4dfcc786b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac7da325d9624a4ea0fcce8620bc9c7a', 'request_id': 'req-5ccfbe55-075a-40b7-8e32-5e4dfcc786b9'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:34.347457 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5ccfbe55-075a-40b7-8e32-5e4dfcc786b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac7da325d9624a4ea0fcce8620bc9c7a', 'request_id': 'req-5ccfbe55-075a-40b7-8e32-5e4dfcc786b9'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:34.355756 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5ccfbe55-075a-40b7-8e32-5e4dfcc786b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac7da325d9624a4ea0fcce8620bc9c7a', 'request_id': 'req-5ccfbe55-075a-40b7-8e32-5e4dfcc786b9'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:34.418522 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5ccfbe55-075a-40b7-8e32-5e4dfcc786b9 admin admin] The token cache is being invalidate because project ac7da325d9624a4ea0fcce8620bc9c7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:06:34.418832 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5ccfbe55-075a-40b7-8e32-5e4dfcc786b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:34.423231 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5ccfbe55-075a-40b7-8e32-5e4dfcc786b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:34.425932 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 826/1658] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:06:33 2022] DELETE /identity/v3/projects/ac7da325d9624a4ea0fcce8620bc9c7a => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:34.431183 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cadeda1-1a6b-4716-a3ff-58a00f19e77d None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:34.431983 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cadeda1-1a6b-4716-a3ff-58a00f19e77d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:34.432384 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cadeda1-1a6b-4716-a3ff-58a00f19e77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:34.466247 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-2cadeda1-1a6b-4716-a3ff-58a00f19e77d None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:34.467698 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2cadeda1-1a6b-4716-a3ff-58a00f19e77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:34.529631 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:34.605587 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 833/1659] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:34.615551 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e654686-225c-485e-b083-ea41abec79b9 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:34.615908 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e654686-225c-485e-b083-ea41abec79b9 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:34.616095 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e654686-225c-485e-b083-ea41abec79b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:34.658145 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1e654686-225c-485e-b083-ea41abec79b9 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:34.661836 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1e654686-225c-485e-b083-ea41abec79b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:34.686386 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 827/1660] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:34.690671 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cadeda1-1a6b-4716-a3ff-58a00f19e77d None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:34.705644 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cadeda1-1a6b-4716-a3ff-58a00f19e77d None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:34.720374 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ed560-cf90-481f-bb98-4f5dd5b9af7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:34.722105 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ed560-cf90-481f-bb98-4f5dd5b9af7d admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:34.722105 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ed560-cf90-481f-bb98-4f5dd5b9af7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:34.722307 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ed560-cf90-481f-bb98-4f5dd5b9af7d admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:34.723076 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3ed560-cf90-481f-bb98-4f5dd5b9af7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:06:34.727814 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3ed560-cf90-481f-bb98-4f5dd5b9af7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:34.730258 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3ed560-cf90-481f-bb98-4f5dd5b9af7d admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:34.763856 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 834/1661] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:06:34.769443 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e654686-225c-485e-b083-ea41abec79b9 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:34.782584 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e654686-225c-485e-b083-ea41abec79b9 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:34.789250 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2bbf9-c9e8-4dd6-84b6-61b3cf770bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:34.790195 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2bbf9-c9e8-4dd6-84b6-61b3cf770bd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:34.790679 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2bbf9-c9e8-4dd6-84b6-61b3cf770bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:34.791331 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2bbf9-c9e8-4dd6-84b6-61b3cf770bd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:34.793311 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a2bbf9-c9e8-4dd6-84b6-61b3cf770bd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:34.795799 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a2bbf9-c9e8-4dd6-84b6-61b3cf770bd7 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:34.912426 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 828/1662] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:06:34 2022] POST /identity/v3/projects => generated 377 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:34.917468 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ed560-cf90-481f-bb98-4f5dd5b9af7d admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:34.930598 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ed560-cf90-481f-bb98-4f5dd5b9af7d admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:35.049386 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b339eb7-9779-4908-8550-de32164b70e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:35.052374 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b339eb7-9779-4908-8550-de32164b70e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:35.054554 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b339eb7-9779-4908-8550-de32164b70e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:35.055028 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b339eb7-9779-4908-8550-de32164b70e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:35.056263 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b339eb7-9779-4908-8550-de32164b70e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:35.061394 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b339eb7-9779-4908-8550-de32164b70e7 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:35.178085 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 835/1663] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:06:34 2022] POST /identity/v3/users => generated 289 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:35.182193 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2bbf9-c9e8-4dd6-84b6-61b3cf770bd7 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:35.220819 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2bbf9-c9e8-4dd6-84b6-61b3cf770bd7 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:35.229462 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc471ea-6320-4b93-8a27-b575fc0e6340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:35.230716 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc471ea-6320-4b93-8a27-b575fc0e6340 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:35.233142 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc471ea-6320-4b93-8a27-b575fc0e6340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:35.233737 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc471ea-6320-4b93-8a27-b575fc0e6340 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:35.235975 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc471ea-6320-4b93-8a27-b575fc0e6340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:35.237727 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc471ea-6320-4b93-8a27-b575fc0e6340 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:35.255612 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 829/1664] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:35.263207 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b339eb7-9779-4908-8550-de32164b70e7 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:35.278931 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b339eb7-9779-4908-8550-de32164b70e7 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:35.290580 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978032ed-1df0-4ce6-8d6b-4ae57e1ee7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:35.291591 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978032ed-1df0-4ce6-8d6b-4ae57e1ee7a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:35.292019 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978032ed-1df0-4ce6-8d6b-4ae57e1ee7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:35.292491 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978032ed-1df0-4ce6-8d6b-4ae57e1ee7a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:35.293315 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978032ed-1df0-4ce6-8d6b-4ae57e1ee7a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:35.296260 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978032ed-1df0-4ce6-8d6b-4ae57e1ee7a7 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:35.328472 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 836/1665] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:35.329515 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc471ea-6320-4b93-8a27-b575fc0e6340 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:35.357423 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc471ea-6320-4b93-8a27-b575fc0e6340 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:35.389143 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c9cbed-6fa8-4928-8d9e-7d01c5fa83b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:35.408113 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c9cbed-6fa8-4928-8d9e-7d01c5fa83b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:35.410423 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c9cbed-6fa8-4928-8d9e-7d01c5fa83b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:35.411872 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c9cbed-6fa8-4928-8d9e-7d01c5fa83b9 admin admin] PATH_INFO: `/v3/projects/b087ee3b57aa43fa8f34151318e982d6/users/51867bd28837452a944628ceefc075f8/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:35.454142 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c9cbed-6fa8-4928-8d9e-7d01c5fa83b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b087ee3b57aa43fa8f34151318e982d6, user_id=51867bd28837452a944628ceefc075f8, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:35.462700 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c9cbed-6fa8-4928-8d9e-7d01c5fa83b9 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:35.637315 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 830/1666] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:06:35 2022] PUT /identity/v3/projects/b087ee3b57aa43fa8f34151318e982d6/users/51867bd28837452a944628ceefc075f8/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:06:35.649638 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df46cca-8759-4589-abc0-8dcc7bb27049 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:35.650646 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df46cca-8759-4589-abc0-8dcc7bb27049 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:35.651068 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df46cca-8759-4589-abc0-8dcc7bb27049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:35.720102 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-0df46cca-8759-4589-abc0-8dcc7bb27049 None None] MFA Rules not processed for user `51867bd28837452a944628ceefc075f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:35.721615 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-0df46cca-8759-4589-abc0-8dcc7bb27049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:35.834997 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 837/1667] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:35.836787 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c9cbed-6fa8-4928-8d9e-7d01c5fa83b9 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:35.997478 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c9cbed-6fa8-4928-8d9e-7d01c5fa83b9 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:36.004758 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c221d4c-b7bd-4619-b9cf-5e24959bff15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:36.006030 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c221d4c-b7bd-4619-b9cf-5e24959bff15 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:36.006427 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c221d4c-b7bd-4619-b9cf-5e24959bff15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:36.007406 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c221d4c-b7bd-4619-b9cf-5e24959bff15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:36.008171 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c221d4c-b7bd-4619-b9cf-5e24959bff15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:36.011789 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c221d4c-b7bd-4619-b9cf-5e24959bff15 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:36.029254 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60228cc8-3c19-4590-badd-58ce5b0682a7 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:36.029555 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60228cc8-3c19-4590-badd-58ce5b0682a7 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:36.029831 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60228cc8-3c19-4590-badd-58ce5b0682a7 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:36.031749 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 838/1668] 10.222.0.27 () {54 vars in 921 bytes} [Sun Sep 4 01:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:06:36.117869 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3be052-f3c3-413d-a9ca-8c9d997be397 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:36.118486 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3be052-f3c3-413d-a9ca-8c9d997be397 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:36.119076 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3be052-f3c3-413d-a9ca-8c9d997be397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:36.136901 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 831/1669] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:06:35 2022] POST /identity/v3/projects => generated 379 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:36.138148 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c221d4c-b7bd-4619-b9cf-5e24959bff15 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:36.153979 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-2f3be052-f3c3-413d-a9ca-8c9d997be397 None None] MFA Rules not processed for user `018802a8d9c041119ad708430f885a89`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:36.156666 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2f3be052-f3c3-413d-a9ca-8c9d997be397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:36.161889 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:36.310198 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c221d4c-b7bd-4619-b9cf-5e24959bff15 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:36.322415 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386109ec-388b-4811-abae-2687a229d30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:36.323078 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386109ec-388b-4811-abae-2687a229d30a admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:36.326006 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386109ec-388b-4811-abae-2687a229d30a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:36.326451 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386109ec-388b-4811-abae-2687a229d30a admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:36.327262 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386109ec-388b-4811-abae-2687a229d30a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:36.331894 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 839/1670] 10.222.0.27 () {58 vars in 1018 bytes} [Sun Sep 4 01:06:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:06:36.335547 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386109ec-388b-4811-abae-2687a229d30a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:36.364310 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:36.484201 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 832/1671] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:06:36 2022] POST /identity/v3/users => generated 296 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:36.492252 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3be052-f3c3-413d-a9ca-8c9d997be397 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:36.505714 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386109ec-388b-4811-abae-2687a229d30a admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:36.507914 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-386109ec-388b-4811-abae-2687a229d30a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:36.511420 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3be052-f3c3-413d-a9ca-8c9d997be397 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:36.558352 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e9cfdb-ac51-4f3c-bd35-a71eef57602e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:36.559189 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9cfdb-ac51-4f3c-bd35-a71eef57602e admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:36.559581 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9cfdb-ac51-4f3c-bd35-a71eef57602e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:36.560254 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9cfdb-ac51-4f3c-bd35-a71eef57602e admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:36.561099 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e9cfdb-ac51-4f3c-bd35-a71eef57602e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:36.579053 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386109ec-388b-4811-abae-2687a229d30a admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:36.579381 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e9cfdb-ac51-4f3c-bd35-a71eef57602e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:36.599670 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc3a54-bea2-48f2-8c7a-17f506c32e49 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '018802a8d9c041119ad708430f885a89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:36.600557 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc3a54-bea2-48f2-8c7a-17f506c32e49 service placement] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:36.608579 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc3a54-bea2-48f2-8c7a-17f506c32e49 service placement] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:36.609126 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc3a54-bea2-48f2-8c7a-17f506c32e49 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:36.617773 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 840/1672] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:36.621123 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e9cfdb-ac51-4f3c-bd35-a71eef57602e admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:36.645953 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e9cfdb-ac51-4f3c-bd35-a71eef57602e admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:36.647996 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bc3a54-bea2-48f2-8c7a-17f506c32e49 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:36.649655 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bc3a54-bea2-48f2-8c7a-17f506c32e49 service placement] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:36.652302 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd33fbe-e3cc-4818-b89e-e231d8fffeb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:36.654149 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd33fbe-e3cc-4818-b89e-e231d8fffeb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:36.655610 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd33fbe-e3cc-4818-b89e-e231d8fffeb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:36.656048 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd33fbe-e3cc-4818-b89e-e231d8fffeb1 admin admin] PATH_INFO: `/v3/projects/d4cdd34edf834802a9e9f83f90f38197/users/33d4d922264d4446b55e789cda2f5df0/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:36.693147 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd33fbe-e3cc-4818-b89e-e231d8fffeb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4cdd34edf834802a9e9f83f90f38197, user_id=33d4d922264d4446b55e789cda2f5df0, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:36.694071 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 833/1673] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:06:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:06:36.701834 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd33fbe-e3cc-4818-b89e-e231d8fffeb1 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:36.819880 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 841/1674] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:06:36 2022] PUT /identity/v3/projects/d4cdd34edf834802a9e9f83f90f38197/users/33d4d922264d4446b55e789cda2f5df0/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:36.822733 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc3a54-bea2-48f2-8c7a-17f506c32e49 service placement] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:36.837413 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc3a54-bea2-48f2-8c7a-17f506c32e49 service placement] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:36.964287 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517888e6-5702-445d-933a-0f76b5c2977a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:36.966369 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517888e6-5702-445d-933a-0f76b5c2977a admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:36.966687 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517888e6-5702-445d-933a-0f76b5c2977a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:36.967585 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517888e6-5702-445d-933a-0f76b5c2977a admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:36.968905 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517888e6-5702-445d-933a-0f76b5c2977a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:36.971365 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517888e6-5702-445d-933a-0f76b5c2977a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:37.014673 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 834/1675] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:37.014917 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd33fbe-e3cc-4818-b89e-e231d8fffeb1 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:37.037311 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd33fbe-e3cc-4818-b89e-e231d8fffeb1 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:37.044597 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de95a61-923c-44d5-bdde-82ec650ca23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:37.044597 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de95a61-923c-44d5-bdde-82ec650ca23a admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:37.044597 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de95a61-923c-44d5-bdde-82ec650ca23a admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:37.044597 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de95a61-923c-44d5-bdde-82ec650ca23a admin admin] PATH_INFO: `/v3/projects/d4cdd34edf834802a9e9f83f90f38197/users/33d4d922264d4446b55e789cda2f5df0/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:37.046206 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de95a61-923c-44d5-bdde-82ec650ca23a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4cdd34edf834802a9e9f83f90f38197, user_id=33d4d922264d4446b55e789cda2f5df0, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:37.051705 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de95a61-923c-44d5-bdde-82ec650ca23a admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:37.085431 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 842/1676] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:06:37 2022] PUT /identity/v3/projects/d4cdd34edf834802a9e9f83f90f38197/users/33d4d922264d4446b55e789cda2f5df0/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:06:37.100140 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66dc854-5321-4363-bfdb-273ed7f8c7ae None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:37.101038 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66dc854-5321-4363-bfdb-273ed7f8c7ae None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:37.101467 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66dc854-5321-4363-bfdb-273ed7f8c7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:37.181230 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-e66dc854-5321-4363-bfdb-273ed7f8c7ae None None] MFA Rules not processed for user `33d4d922264d4446b55e789cda2f5df0`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:37.182847 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e66dc854-5321-4363-bfdb-273ed7f8c7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:37.397362 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 835/1677] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:37 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:37.410835 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de95a61-923c-44d5-bdde-82ec650ca23a admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:37.412606 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-6de95a61-923c-44d5-bdde-82ec650ca23a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:37.544813 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de95a61-923c-44d5-bdde-82ec650ca23a admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:37.549220 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c30f0-5b43-4499-a900-9f3b4f47c35a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:37.550400 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c30f0-5b43-4499-a900-9f3b4f47c35a service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:37.550686 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c30f0-5b43-4499-a900-9f3b4f47c35a service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:37.551057 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c30f0-5b43-4499-a900-9f3b4f47c35a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:37.785916 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6c30f0-5b43-4499-a900-9f3b4f47c35a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:37.789265 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6c30f0-5b43-4499-a900-9f3b4f47c35a service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:37.817203 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 843/1678] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:06:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 01:06:38.047651 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66dc854-5321-4363-bfdb-273ed7f8c7ae None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:38.050601 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e66dc854-5321-4363-bfdb-273ed7f8c7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:38.200324 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66dc854-5321-4363-bfdb-273ed7f8c7ae None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:38.246857 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4480080d-f600-4296-83a1-7701a797911a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:38.247873 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480080d-f600-4296-83a1-7701a797911a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:38.248376 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480080d-f600-4296-83a1-7701a797911a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:38.249931 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480080d-f600-4296-83a1-7701a797911a admin admin] PATH_INFO: `/v3/users/b5e4b2720c0645b6b8c20974f28412ea` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:38.252185 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4480080d-f600-4296-83a1-7701a797911a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5e4b2720c0645b6b8c20974f28412ea)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:38.258960 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4480080d-f600-4296-83a1-7701a797911a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:38.503895 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4480080d-f600-4296-83a1-7701a797911a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5e4b2720c0645b6b8c20974f28412ea'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:38.503895 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4480080d-f600-4296-83a1-7701a797911a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5e4b2720c0645b6b8c20974f28412ea'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:38.527287 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4480080d-f600-4296-83a1-7701a797911a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5e4b2720c0645b6b8c20974f28412ea'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:38.609180 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4480080d-f600-4296-83a1-7701a797911a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5e4b2720c0645b6b8c20974f28412ea'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:38.609838 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4480080d-f600-4296-83a1-7701a797911a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5e4b2720c0645b6b8c20974f28412ea'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:38.653009 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a28fb3-0249-4bcc-a6f8-9128ad030505 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:38.653294 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a28fb3-0249-4bcc-a6f8-9128ad030505 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:38.654007 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a28fb3-0249-4bcc-a6f8-9128ad030505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:38.661978 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 836/1679] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:06:38 2022] DELETE /identity/v3/users/b5e4b2720c0645b6b8c20974f28412ea => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:38.691520 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-d3a28fb3-0249-4bcc-a6f8-9128ad030505 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:38.692846 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d3a28fb3-0249-4bcc-a6f8-9128ad030505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:38.775917 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:38.869653 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:38.872368 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 844/1680] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:39.054764 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4480080d-f600-4296-83a1-7701a797911a admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:39.056663 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4480080d-f600-4296-83a1-7701a797911a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:39.187899 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d72f8-cc80-4534-9f0c-a6fc46e78688 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:39.188218 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d72f8-cc80-4534-9f0c-a6fc46e78688 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:39.188516 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d72f8-cc80-4534-9f0c-a6fc46e78688 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:39.190342 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 845/1681] 10.222.0.27 () {54 vars in 921 bytes} [Sun Sep 4 01:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:06:39.203530 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4480080d-f600-4296-83a1-7701a797911a admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:39.212765 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314db425-fa38-405f-b56b-e2083909b8f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:39.213811 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314db425-fa38-405f-b56b-e2083909b8f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:39.214288 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314db425-fa38-405f-b56b-e2083909b8f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:39.215307 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314db425-fa38-405f-b56b-e2083909b8f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:39.247802 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314db425-fa38-405f-b56b-e2083909b8f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:39.249327 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314db425-fa38-405f-b56b-e2083909b8f4 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:39.272449 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3fe5e4-17c1-4087-96fc-e2ffec391676 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:39.272955 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3fe5e4-17c1-4087-96fc-e2ffec391676 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:39.273402 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3fe5e4-17c1-4087-96fc-e2ffec391676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:39.289325 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 837/1682] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:06:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:06:39.336003 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-0e3fe5e4-17c1-4087-96fc-e2ffec391676 None None] MFA Rules not processed for user `018802a8d9c041119ad708430f885a89`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:39.337591 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-0e3fe5e4-17c1-4087-96fc-e2ffec391676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:39.477437 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 846/1683] 10.222.0.27 () {58 vars in 1018 bytes} [Sun Sep 4 01:06:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:06:39.531239 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314db425-fa38-405f-b56b-e2083909b8f4 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:39.532929 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-314db425-fa38-405f-b56b-e2083909b8f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:39.541701 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3fe5e4-17c1-4087-96fc-e2ffec391676 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:39.544238 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-0e3fe5e4-17c1-4087-96fc-e2ffec391676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:39.570492 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314db425-fa38-405f-b56b-e2083909b8f4 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:39.576580 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3fe5e4-17c1-4087-96fc-e2ffec391676 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:39.591422 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d207622-8073-48e6-b7c6-c46ceb52c0be service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '018802a8d9c041119ad708430f885a89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:39.592076 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d207622-8073-48e6-b7c6-c46ceb52c0be service placement] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:39.592354 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d207622-8073-48e6-b7c6-c46ceb52c0be service placement] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:39.592642 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d207622-8073-48e6-b7c6-c46ceb52c0be service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:39.617650 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f4c6f-e23d-460d-aeb2-c5a3ae50a9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:39.617822 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d207622-8073-48e6-b7c6-c46ceb52c0be service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:39.618921 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d207622-8073-48e6-b7c6-c46ceb52c0be service placement] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:39.627665 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f4c6f-e23d-460d-aeb2-c5a3ae50a9e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:39.628282 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f4c6f-e23d-460d-aeb2-c5a3ae50a9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:39.628604 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f4c6f-e23d-460d-aeb2-c5a3ae50a9e6 admin admin] PATH_INFO: `/v3/users/636acaae4bcb43e5bb7a3205e816d0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:39.630295 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209f4c6f-e23d-460d-aeb2-c5a3ae50a9e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=636acaae4bcb43e5bb7a3205e816d0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:39.632620 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209f4c6f-e23d-460d-aeb2-c5a3ae50a9e6 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:39.646751 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 847/1684] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:06:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:06:39.893860 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-209f4c6f-e23d-460d-aeb2-c5a3ae50a9e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '636acaae4bcb43e5bb7a3205e816d0be'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:39.894502 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-209f4c6f-e23d-460d-aeb2-c5a3ae50a9e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '636acaae4bcb43e5bb7a3205e816d0be'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:39.917246 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-209f4c6f-e23d-460d-aeb2-c5a3ae50a9e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '636acaae4bcb43e5bb7a3205e816d0be'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:40.191382 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-209f4c6f-e23d-460d-aeb2-c5a3ae50a9e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '636acaae4bcb43e5bb7a3205e816d0be'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:40.192098 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-209f4c6f-e23d-460d-aeb2-c5a3ae50a9e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '636acaae4bcb43e5bb7a3205e816d0be'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:40.243737 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 838/1685] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:06:39 2022] DELETE /identity/v3/users/636acaae4bcb43e5bb7a3205e816d0be => generated 0 bytes in 713 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:06:40.455690 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b8af9-aa8d-4192-b788-028c51958f44 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:40.457661 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b8af9-aa8d-4192-b788-028c51958f44 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:40.459835 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b8af9-aa8d-4192-b788-028c51958f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:40.561417 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-ae5b8af9-aa8d-4192-b788-028c51958f44 None None] MFA Rules not processed for user `bd1c41fcdc724a128117a5288b8559f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:40.563034 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ae5b8af9-aa8d-4192-b788-028c51958f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:40.743097 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 848/1686] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:40 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:40.761032 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f4c6f-e23d-460d-aeb2-c5a3ae50a9e6 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:40.768779 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b8af9-aa8d-4192-b788-028c51958f44 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:40.771560 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-209f4c6f-e23d-460d-aeb2-c5a3ae50a9e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:40.774061 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ae5b8af9-aa8d-4192-b788-028c51958f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:40.863546 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:40.971775 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:41.012634 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f4c6f-e23d-460d-aeb2-c5a3ae50a9e6 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:41.038266 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92abaf0a-1023-4663-9e59-58541d18a004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:41.045587 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abaf0a-1023-4663-9e59-58541d18a004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:41.047022 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abaf0a-1023-4663-9e59-58541d18a004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:41.047310 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abaf0a-1023-4663-9e59-58541d18a004 admin admin] PATH_INFO: `/v3/projects/5a390896075b4d7fb32eb2bfd80f2431` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:41.055592 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92abaf0a-1023-4663-9e59-58541d18a004 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a390896075b4d7fb32eb2bfd80f2431)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:41.066559 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92abaf0a-1023-4663-9e59-58541d18a004 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:41.069993 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b8af9-aa8d-4192-b788-028c51958f44 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:41.088505 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d67375b-a380-4d5d-ad92-2e950db438e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:41.089767 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67375b-a380-4d5d-ad92-2e950db438e7 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:41.090342 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67375b-a380-4d5d-ad92-2e950db438e7 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:41.091974 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67375b-a380-4d5d-ad92-2e950db438e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:41.125612 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d67375b-a380-4d5d-ad92-2e950db438e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:41.128145 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d67375b-a380-4d5d-ad92-2e950db438e7 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:41.159923 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 849/1687] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:06:40 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:06:41.161404 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d67375b-a380-4d5d-ad92-2e950db438e7 service nova] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:41.165510 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-7d67375b-a380-4d5d-ad92-2e950db438e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:41.211516 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d67375b-a380-4d5d-ad92-2e950db438e7 service nova] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:41.260582 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-92abaf0a-1023-4663-9e59-58541d18a004 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a390896075b4d7fb32eb2bfd80f2431', 'request_id': 'req-92abaf0a-1023-4663-9e59-58541d18a004'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:41.365843 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-92abaf0a-1023-4663-9e59-58541d18a004 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a390896075b4d7fb32eb2bfd80f2431', 'request_id': 'req-92abaf0a-1023-4663-9e59-58541d18a004'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:41.375809 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-92abaf0a-1023-4663-9e59-58541d18a004 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a390896075b4d7fb32eb2bfd80f2431', 'request_id': 'req-92abaf0a-1023-4663-9e59-58541d18a004'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:41.413971 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-92abaf0a-1023-4663-9e59-58541d18a004 admin admin] The token cache is being invalidate because project 5a390896075b4d7fb32eb2bfd80f2431 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:06:41.424012 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-92abaf0a-1023-4663-9e59-58541d18a004 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:41.427226 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-92abaf0a-1023-4663-9e59-58541d18a004 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:41.434515 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 839/1688] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:06:40 2022] DELETE /identity/v3/projects/5a390896075b4d7fb32eb2bfd80f2431 => generated 0 bytes in 674 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:41.442437 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92abaf0a-1023-4663-9e59-58541d18a004 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:41.453146 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-92abaf0a-1023-4663-9e59-58541d18a004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:41.480702 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e9724-df81-421b-aa02-c4a75ac8ec28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:41.482293 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e9724-df81-421b-aa02-c4a75ac8ec28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:41.488980 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e9724-df81-421b-aa02-c4a75ac8ec28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:41.497444 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e9724-df81-421b-aa02-c4a75ac8ec28 admin admin] PATH_INFO: `/v3/projects/dcc7379edc954077ab3bf936802fd29f` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:41.499687 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e9724-df81-421b-aa02-c4a75ac8ec28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcc7379edc954077ab3bf936802fd29f)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:41.501689 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e9724-df81-421b-aa02-c4a75ac8ec28 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:41.693011 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92abaf0a-1023-4663-9e59-58541d18a004 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:41.737346 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-391e9724-df81-421b-aa02-c4a75ac8ec28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcc7379edc954077ab3bf936802fd29f', 'request_id': 'req-391e9724-df81-421b-aa02-c4a75ac8ec28'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:41.843286 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-391e9724-df81-421b-aa02-c4a75ac8ec28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcc7379edc954077ab3bf936802fd29f', 'request_id': 'req-391e9724-df81-421b-aa02-c4a75ac8ec28'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:41.854743 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-391e9724-df81-421b-aa02-c4a75ac8ec28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcc7379edc954077ab3bf936802fd29f', 'request_id': 'req-391e9724-df81-421b-aa02-c4a75ac8ec28'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:41.891753 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-391e9724-df81-421b-aa02-c4a75ac8ec28 admin admin] The token cache is being invalidate because project dcc7379edc954077ab3bf936802fd29f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:06:41.892696 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-391e9724-df81-421b-aa02-c4a75ac8ec28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:41.901294 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-391e9724-df81-421b-aa02-c4a75ac8ec28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:41.906872 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934ae12-a377-4724-b964-f11738b31b43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:41.912262 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934ae12-a377-4724-b964-f11738b31b43 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:41.912393 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 850/1689] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:06:41 2022] DELETE /identity/v3/projects/dcc7379edc954077ab3bf936802fd29f => generated 0 bytes in 747 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:41.912525 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934ae12-a377-4724-b964-f11738b31b43 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:41.912525 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934ae12-a377-4724-b964-f11738b31b43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:41.912675 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2bab61-7fc4-48c5-92a4-bb24e7bfecff None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:41.913238 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2bab61-7fc4-48c5-92a4-bb24e7bfecff None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:41.913238 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2bab61-7fc4-48c5-92a4-bb24e7bfecff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:41.917835 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4934ae12-a377-4724-b964-f11738b31b43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:41.946606 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-ce2bab61-7fc4-48c5-92a4-bb24e7bfecff None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:41.948823 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ce2bab61-7fc4-48c5-92a4-bb24e7bfecff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:42.121688 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:42.132371 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934ae12-a377-4724-b964-f11738b31b43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:42.140676 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934ae12-a377-4724-b964-f11738b31b43 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:42.150128 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 851/1690] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:42.150128 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2bab61-7fc4-48c5-92a4-bb24e7bfecff None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:42.150128 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ce2bab61-7fc4-48c5-92a4-bb24e7bfecff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:42.171834 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 840/1691] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:06:41 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 736 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:06:42.173065 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934ae12-a377-4724-b964-f11738b31b43 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:42.176638 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4934ae12-a377-4724-b964-f11738b31b43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:42.191077 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2bab61-7fc4-48c5-92a4-bb24e7bfecff None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:42.203167 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b725ddce-15a1-42de-ac6f-a91edf3e28d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:42.206982 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725ddce-15a1-42de-ac6f-a91edf3e28d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:42.209106 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725ddce-15a1-42de-ac6f-a91edf3e28d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:42.209648 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725ddce-15a1-42de-ac6f-a91edf3e28d8 admin admin] PATH_INFO: `/v3/users/014ce9fd70c0440ba5973239f9c496a9` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:42.212308 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b725ddce-15a1-42de-ac6f-a91edf3e28d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=014ce9fd70c0440ba5973239f9c496a9)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:42.227047 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b725ddce-15a1-42de-ac6f-a91edf3e28d8 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:42.227502 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934ae12-a377-4724-b964-f11738b31b43 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:42.242717 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcdb838-c8c4-46c0-9a7b-fb21353c756a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:42.247280 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcdb838-c8c4-46c0-9a7b-fb21353c756a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:42.247751 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcdb838-c8c4-46c0-9a7b-fb21353c756a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:42.248135 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcdb838-c8c4-46c0-9a7b-fb21353c756a admin admin] PATH_INFO: `/v3/users/0cabbafc1e8b4263b740017ead3a1f8d` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:42.250213 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcdb838-c8c4-46c0-9a7b-fb21353c756a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cabbafc1e8b4263b740017ead3a1f8d)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:42.256242 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcdb838-c8c4-46c0-9a7b-fb21353c756a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:42.502880 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b725ddce-15a1-42de-ac6f-a91edf3e28d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '014ce9fd70c0440ba5973239f9c496a9'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:42.505422 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b725ddce-15a1-42de-ac6f-a91edf3e28d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '014ce9fd70c0440ba5973239f9c496a9'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:42.568410 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-ddcdb838-c8c4-46c0-9a7b-fb21353c756a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cabbafc1e8b4263b740017ead3a1f8d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:42.570080 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-ddcdb838-c8c4-46c0-9a7b-fb21353c756a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cabbafc1e8b4263b740017ead3a1f8d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:42.587595 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-ddcdb838-c8c4-46c0-9a7b-fb21353c756a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cabbafc1e8b4263b740017ead3a1f8d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:42.612575 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:42.613159 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b725ddce-15a1-42de-ac6f-a91edf3e28d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '014ce9fd70c0440ba5973239f9c496a9'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:42.614701 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b725ddce-15a1-42de-ac6f-a91edf3e28d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '014ce9fd70c0440ba5973239f9c496a9'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:42.675613 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b725ddce-15a1-42de-ac6f-a91edf3e28d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '014ce9fd70c0440ba5973239f9c496a9'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:42.762902 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 852/1692] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:06:42 2022] DELETE /identity/v3/users/014ce9fd70c0440ba5973239f9c496a9 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:42.762902 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c3684-58a8-4167-af62-54daa93fe1b2 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:42.762902 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c3684-58a8-4167-af62-54daa93fe1b2 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:42.762902 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c3684-58a8-4167-af62-54daa93fe1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:42.812135 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-8d3c3684-58a8-4167-af62-54daa93fe1b2 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:42.813303 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8d3c3684-58a8-4167-af62-54daa93fe1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:42.855159 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-ddcdb838-c8c4-46c0-9a7b-fb21353c756a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cabbafc1e8b4263b740017ead3a1f8d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:42.856105 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-ddcdb838-c8c4-46c0-9a7b-fb21353c756a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cabbafc1e8b4263b740017ead3a1f8d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:42.899693 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 841/1693] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:06:42 2022] DELETE /identity/v3/users/0cabbafc1e8b4263b740017ead3a1f8d => generated 0 bytes in 721 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:42.908367 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e58a3-145b-4d2f-b5b3-b5a272eeed16 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:42.914704 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e58a3-145b-4d2f-b5b3-b5a272eeed16 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:42.914993 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e58a3-145b-4d2f-b5b3-b5a272eeed16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:42.995463 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 853/1694] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:42.998212 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6463bae-bbc2-4631-8b84-2f00ea7ab42d None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:42.999109 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6463bae-bbc2-4631-8b84-2f00ea7ab42d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:42.999491 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6463bae-bbc2-4631-8b84-2f00ea7ab42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:43.075166 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-c6463bae-bbc2-4631-8b84-2f00ea7ab42d None None] MFA Rules not processed for user `9bef8dc1da4c44dbbafcaf3c813b3262`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:43.076797 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c6463bae-bbc2-4631-8b84-2f00ea7ab42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:43.084761 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-a69e58a3-145b-4d2f-b5b3-b5a272eeed16 None None] MFA Rules not processed for user `0c5e9731f6204b44841b8524762d5a77`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:43.086331 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a69e58a3-145b-4d2f-b5b3-b5a272eeed16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:43.229778 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 854/1695] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:43.231116 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48da64c-2b86-4c30-ae91-a5e6fc017001 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:43.231408 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48da64c-2b86-4c30-ae91-a5e6fc017001 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:43.231690 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48da64c-2b86-4c30-ae91-a5e6fc017001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:43.292293 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 842/1696] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:43.297554 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-d48da64c-2b86-4c30-ae91-a5e6fc017001 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:43.298807 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d48da64c-2b86-4c30-ae91-a5e6fc017001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:43.306623 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e58a3-145b-4d2f-b5b3-b5a272eeed16 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:43.466396 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 855/1697] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:43.472194 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779a14d-8132-4949-b8ba-1029501319dc None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:43.472499 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779a14d-8132-4949-b8ba-1029501319dc None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:43.472992 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779a14d-8132-4949-b8ba-1029501319dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:43.474803 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e58a3-145b-4d2f-b5b3-b5a272eeed16 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:43.489192 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38559eb-b1fd-459b-81b1-32dde3c87d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:43.489837 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38559eb-b1fd-459b-81b1-32dde3c87d67 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:43.490121 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38559eb-b1fd-459b-81b1-32dde3c87d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:43.490398 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38559eb-b1fd-459b-81b1-32dde3c87d67 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:43.491070 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38559eb-b1fd-459b-81b1-32dde3c87d67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:06:43.491380 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38559eb-b1fd-459b-81b1-32dde3c87d67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:43.493923 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38559eb-b1fd-459b-81b1-32dde3c87d67 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:43.517588 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-d779a14d-8132-4949-b8ba-1029501319dc None None] MFA Rules not processed for user `9bef8dc1da4c44dbbafcaf3c813b3262`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:43.519631 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d779a14d-8132-4949-b8ba-1029501319dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:43.527707 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 843/1698] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:06:43.528966 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38559eb-b1fd-459b-81b1-32dde3c87d67 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:43.532104 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e38559eb-b1fd-459b-81b1-32dde3c87d67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:43.551990 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 856/1699] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:43 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:43.552837 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779a14d-8132-4949-b8ba-1029501319dc None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:43.582719 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779a14d-8132-4949-b8ba-1029501319dc None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:43.588673 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8cf8b-cec1-4be1-9df1-cd2cf44ff63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:43.589311 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8cf8b-cec1-4be1-9df1-cd2cf44ff63b admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:43.589594 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8cf8b-cec1-4be1-9df1-cd2cf44ff63b admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:43.589866 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8cf8b-cec1-4be1-9df1-cd2cf44ff63b admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:43.590503 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f8cf8b-cec1-4be1-9df1-cd2cf44ff63b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:43.592587 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f8cf8b-cec1-4be1-9df1-cd2cf44ff63b admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:43.709139 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38559eb-b1fd-459b-81b1-32dde3c87d67 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:43.719487 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3536d3-d871-4b66-b640-d862411a9774 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:43.720125 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3536d3-d871-4b66-b640-d862411a9774 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:43.720408 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3536d3-d871-4b66-b640-d862411a9774 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:43.720701 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3536d3-d871-4b66-b640-d862411a9774 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:43.737914 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3536d3-d871-4b66-b640-d862411a9774 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:43.737914 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3536d3-d871-4b66-b640-d862411a9774 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:43.751534 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 844/1700] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:06:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:06:43.754303 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3536d3-d871-4b66-b640-d862411a9774 service nova] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:43.755937 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9c3536d3-d871-4b66-b640-d862411a9774 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:43.768893 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 857/1701] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:06:43 2022] POST /identity/v3/projects => generated 371 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:43.770472 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb0f1f-2e23-43b0-aa9b-226070785adb None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:43.773766 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb0f1f-2e23-43b0-aa9b-226070785adb None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:43.774856 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb0f1f-2e23-43b0-aa9b-226070785adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:43.829909 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3536d3-d871-4b66-b640-d862411a9774 service nova] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:43.878002 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-21bb0f1f-2e23-43b0-aa9b-226070785adb None None] MFA Rules not processed for user `a28ff717beef47189f1d005fe7f65a10`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:43.878002 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-21bb0f1f-2e23-43b0-aa9b-226070785adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:43.982872 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e044a7-4b7b-4e63-91a1-32ac8a63ffef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:43.983904 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e044a7-4b7b-4e63-91a1-32ac8a63ffef service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:43.984402 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e044a7-4b7b-4e63-91a1-32ac8a63ffef service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:43.984940 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e044a7-4b7b-4e63-91a1-32ac8a63ffef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:44.057680 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:44.116752 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 858/1702] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:44.117634 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb0f1f-2e23-43b0-aa9b-226070785adb None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:44.130071 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-21bb0f1f-2e23-43b0-aa9b-226070785adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:44.161033 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e044a7-4b7b-4e63-91a1-32ac8a63ffef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:44.163019 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e044a7-4b7b-4e63-91a1-32ac8a63ffef service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:44.207043 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 845/1703] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:06:43 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:06:44.211481 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e044a7-4b7b-4e63-91a1-32ac8a63ffef service nova] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:44.336139 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb0f1f-2e23-43b0-aa9b-226070785adb None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:44.348110 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04820a05-7fff-4ddf-a98c-51ab0bb9eaf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:44.356165 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04820a05-7fff-4ddf-a98c-51ab0bb9eaf0 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:44.356361 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04820a05-7fff-4ddf-a98c-51ab0bb9eaf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:44.356671 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04820a05-7fff-4ddf-a98c-51ab0bb9eaf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:44.424241 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e044a7-4b7b-4e63-91a1-32ac8a63ffef service nova] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:44.438364 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39533f05-fe75-45ff-86e6-2a80adddb390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:44.440068 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39533f05-fe75-45ff-86e6-2a80adddb390 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:44.441349 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39533f05-fe75-45ff-86e6-2a80adddb390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:44.443380 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39533f05-fe75-45ff-86e6-2a80adddb390 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:44.444285 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39533f05-fe75-45ff-86e6-2a80adddb390 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:44.453679 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39533f05-fe75-45ff-86e6-2a80adddb390 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:44.535475 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:44.543084 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04820a05-7fff-4ddf-a98c-51ab0bb9eaf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:44.544715 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04820a05-7fff-4ddf-a98c-51ab0bb9eaf0 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:44.576355 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 859/1704] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:06:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:06:44.576355 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04820a05-7fff-4ddf-a98c-51ab0bb9eaf0 service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:44.577146 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-04820a05-7fff-4ddf-a98c-51ab0bb9eaf0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:44.601861 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04820a05-7fff-4ddf-a98c-51ab0bb9eaf0 service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:44.613143 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e4167-5e43-491f-b23c-8d4a1088a636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:44.613800 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e4167-5e43-491f-b23c-8d4a1088a636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:44.614075 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e4167-5e43-491f-b23c-8d4a1088a636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:44.614919 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e4167-5e43-491f-b23c-8d4a1088a636 admin admin] PATH_INFO: `/v3/projects/a7f4995ce2c443a6924b53184dc2df32` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:44.618736 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238e4167-5e43-491f-b23c-8d4a1088a636 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7f4995ce2c443a6924b53184dc2df32)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:44.621402 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 846/1705] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:06:44 2022] POST /identity/v3/users => generated 286 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:44.621542 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238e4167-5e43-491f-b23c-8d4a1088a636 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:44.622489 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560929cd-fca0-4e51-9e71-6b75342abfe8 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:44.622897 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560929cd-fca0-4e51-9e71-6b75342abfe8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:44.623179 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560929cd-fca0-4e51-9e71-6b75342abfe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:44.693865 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-560929cd-fca0-4e51-9e71-6b75342abfe8 None None] MFA Rules not processed for user `a28ff717beef47189f1d005fe7f65a10`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:44.695614 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-560929cd-fca0-4e51-9e71-6b75342abfe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:44.724448 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 847/1706] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:44.733683 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560929cd-fca0-4e51-9e71-6b75342abfe8 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:44.763229 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560929cd-fca0-4e51-9e71-6b75342abfe8 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:44.781074 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-238e4167-5e43-491f-b23c-8d4a1088a636 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7f4995ce2c443a6924b53184dc2df32', 'request_id': 'req-238e4167-5e43-491f-b23c-8d4a1088a636'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:44.865957 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab71fbd8-7e24-48f2-8750-cc3f302ac934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:44.866392 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-238e4167-5e43-491f-b23c-8d4a1088a636 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7f4995ce2c443a6924b53184dc2df32', 'request_id': 'req-238e4167-5e43-491f-b23c-8d4a1088a636'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:44.866568 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71fbd8-7e24-48f2-8750-cc3f302ac934 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:44.866853 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71fbd8-7e24-48f2-8750-cc3f302ac934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:44.867122 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71fbd8-7e24-48f2-8750-cc3f302ac934 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:44.867764 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab71fbd8-7e24-48f2-8750-cc3f302ac934 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:44.869410 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab71fbd8-7e24-48f2-8750-cc3f302ac934 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:44.873592 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-238e4167-5e43-491f-b23c-8d4a1088a636 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7f4995ce2c443a6924b53184dc2df32', 'request_id': 'req-238e4167-5e43-491f-b23c-8d4a1088a636'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:44.886710 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 848/1707] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:44.887063 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab71fbd8-7e24-48f2-8750-cc3f302ac934 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:44.896550 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-238e4167-5e43-491f-b23c-8d4a1088a636 admin admin] The token cache is being invalidate because project a7f4995ce2c443a6924b53184dc2df32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:06:44.896905 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-238e4167-5e43-491f-b23c-8d4a1088a636 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:44.899714 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-238e4167-5e43-491f-b23c-8d4a1088a636 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:44.903912 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 860/1708] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:06:44 2022] DELETE /identity/v3/projects/a7f4995ce2c443a6924b53184dc2df32 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:44.906062 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e4167-5e43-491f-b23c-8d4a1088a636 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:44.910079 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-238e4167-5e43-491f-b23c-8d4a1088a636 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:44.955432 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e4167-5e43-491f-b23c-8d4a1088a636 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:44.974658 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6382b57e-62df-4587-a8e3-152a00fc12f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:44.978997 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382b57e-62df-4587-a8e3-152a00fc12f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:44.978997 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382b57e-62df-4587-a8e3-152a00fc12f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:44.978997 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382b57e-62df-4587-a8e3-152a00fc12f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:44.980926 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6382b57e-62df-4587-a8e3-152a00fc12f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:44.982693 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6382b57e-62df-4587-a8e3-152a00fc12f0 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:45.024141 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 861/1709] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:45.028257 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cf58a-bb5c-47d9-83d2-b527ceac3714 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:45.028595 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cf58a-bb5c-47d9-83d2-b527ceac3714 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:45.028849 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cf58a-bb5c-47d9-83d2-b527ceac3714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:45.082517 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab71fbd8-7e24-48f2-8750-cc3f302ac934 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:45.094785 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-666cf58a-bb5c-47d9-83d2-b527ceac3714 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:45.099145 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-666cf58a-bb5c-47d9-83d2-b527ceac3714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:45.099710 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8683a24a-6f2e-4b99-8851-3ddde2c06946 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:45.100357 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683a24a-6f2e-4b99-8851-3ddde2c06946 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:45.100679 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683a24a-6f2e-4b99-8851-3ddde2c06946 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:45.103210 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683a24a-6f2e-4b99-8851-3ddde2c06946 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:45.108203 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8683a24a-6f2e-4b99-8851-3ddde2c06946 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:45.120218 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 862/1710] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:45.121244 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666cf58a-bb5c-47d9-83d2-b527ceac3714 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:45.160052 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666cf58a-bb5c-47d9-83d2-b527ceac3714 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:45.170970 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2e08b-42d6-4ca2-9c7e-3d8ddc078feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:45.171470 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2e08b-42d6-4ca2-9c7e-3d8ddc078feb admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:45.171753 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2e08b-42d6-4ca2-9c7e-3d8ddc078feb admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:45.172029 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2e08b-42d6-4ca2-9c7e-3d8ddc078feb admin admin] PATH_INFO: `/v3/projects/2a59f67920fb433c8018ec4d98a87b8a/users/128e8929aef44054b4aa4e4dc7d13f20/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:45.193378 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2e08b-42d6-4ca2-9c7e-3d8ddc078feb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a59f67920fb433c8018ec4d98a87b8a, user_id=128e8929aef44054b4aa4e4dc7d13f20, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:45.197295 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2e08b-42d6-4ca2-9c7e-3d8ddc078feb admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:45.241701 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8683a24a-6f2e-4b99-8851-3ddde2c06946 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:45.243738 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8683a24a-6f2e-4b99-8851-3ddde2c06946 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:45.254439 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 849/1711] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:06:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:06:45.256053 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0bddfa-8ea6-4dd8-8251-0e036e585827 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:45.256590 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0bddfa-8ea6-4dd8-8251-0e036e585827 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:45.257028 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0bddfa-8ea6-4dd8-8251-0e036e585827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:45.286921 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-db0bddfa-8ea6-4dd8-8251-0e036e585827 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:45.289094 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-db0bddfa-8ea6-4dd8-8251-0e036e585827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:45.310999 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 863/1712] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:06:45 2022] PUT /identity/v3/projects/2a59f67920fb433c8018ec4d98a87b8a/users/128e8929aef44054b4aa4e4dc7d13f20/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:45.310999 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03788091-978d-4ccd-adc8-032f2ea5f3e6 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:45.310999 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03788091-978d-4ccd-adc8-032f2ea5f3e6 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:45.310999 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03788091-978d-4ccd-adc8-032f2ea5f3e6 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:45.313932 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 864/1713] 10.222.0.27 () {54 vars in 912 bytes} [Sun Sep 4 01:06:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:06:45.320871 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbb05d-3b04-4b6e-894e-a624f8c56a40 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:45.321158 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 850/1714] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:45.324929 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbb05d-3b04-4b6e-894e-a624f8c56a40 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:45.325237 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbb05d-3b04-4b6e-894e-a624f8c56a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:45.326311 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0bddfa-8ea6-4dd8-8251-0e036e585827 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:45.355056 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0bddfa-8ea6-4dd8-8251-0e036e585827 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:45.443514 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-dafbb05d-3b04-4b6e-894e-a624f8c56a40 None None] MFA Rules not processed for user `128e8929aef44054b4aa4e4dc7d13f20`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:45.444799 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-dafbb05d-3b04-4b6e-894e-a624f8c56a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:45.530734 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3cff85-163d-4e79-8590-5fb526290625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:45.535648 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3cff85-163d-4e79-8590-5fb526290625 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:45.535967 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3cff85-163d-4e79-8590-5fb526290625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:45.536249 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3cff85-163d-4e79-8590-5fb526290625 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:45.536976 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3cff85-163d-4e79-8590-5fb526290625 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:06:45.537479 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3cff85-163d-4e79-8590-5fb526290625 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:45.541292 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3cff85-163d-4e79-8590-5fb526290625 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:45.572929 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 851/1715] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:06:45.575943 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6899e31-1fdf-4c88-a7db-d8d40681fe03 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:45.576244 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6899e31-1fdf-4c88-a7db-d8d40681fe03 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:45.576561 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6899e31-1fdf-4c88-a7db-d8d40681fe03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:45.621900 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 865/1716] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:45.628898 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafbb05d-3b04-4b6e-894e-a624f8c56a40 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:45.654992 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafbb05d-3b04-4b6e-894e-a624f8c56a40 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:45.670284 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956537b-0a00-4538-974b-86602f598ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:45.671015 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956537b-0a00-4538-974b-86602f598ff4 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:45.671365 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956537b-0a00-4538-974b-86602f598ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:45.671715 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956537b-0a00-4538-974b-86602f598ff4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:45.672963 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956537b-0a00-4538-974b-86602f598ff4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:45.675359 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956537b-0a00-4538-974b-86602f598ff4 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:45.685594 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f6899e31-1fdf-4c88-a7db-d8d40681fe03 None None] MFA Rules not processed for user `861c278f8bf34b3bb407b2e051119ee9`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:45.686875 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f6899e31-1fdf-4c88-a7db-d8d40681fe03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:45.827974 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:45.829126 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 852/1717] 10.222.0.27 () {58 vars in 1009 bytes} [Sun Sep 4 01:06:45 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:45.840381 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 866/1718] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:06:45 2022] POST /identity/v3/projects => generated 357 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:45.845091 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6899e31-1fdf-4c88-a7db-d8d40681fe03 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:45.860256 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6899e31-1fdf-4c88-a7db-d8d40681fe03 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:45.869835 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956537b-0a00-4538-974b-86602f598ff4 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:45.869835 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c956537b-0a00-4538-974b-86602f598ff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:46.038712 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1d246d-4b54-4582-af1b-cbadc542b337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:46.039361 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d246d-4b54-4582-af1b-cbadc542b337 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:46.039918 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d246d-4b54-4582-af1b-cbadc542b337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:46.040312 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d246d-4b54-4582-af1b-cbadc542b337 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:46.040996 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1d246d-4b54-4582-af1b-cbadc542b337 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:46.043077 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1d246d-4b54-4582-af1b-cbadc542b337 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:46.047591 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956537b-0a00-4538-974b-86602f598ff4 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:46.054036 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9071d1eb-a1fb-458d-a0fc-cb322da388ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:46.054722 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071d1eb-a1fb-458d-a0fc-cb322da388ad service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:46.055133 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071d1eb-a1fb-458d-a0fc-cb322da388ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:46.055588 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071d1eb-a1fb-458d-a0fc-cb322da388ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:46.069735 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9071d1eb-a1fb-458d-a0fc-cb322da388ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:46.069735 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9071d1eb-a1fb-458d-a0fc-cb322da388ad service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:46.077637 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 867/1719] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:06:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 01:06:46.086969 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9071d1eb-a1fb-458d-a0fc-cb322da388ad service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:46.092495 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-9071d1eb-a1fb-458d-a0fc-cb322da388ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:46.145890 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9071d1eb-a1fb-458d-a0fc-cb322da388ad service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:46.161168 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a78c45-0ade-43a7-bb91-4cbbb065df57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:46.162246 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a78c45-0ade-43a7-bb91-4cbbb065df57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:46.162735 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a78c45-0ade-43a7-bb91-4cbbb065df57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:46.163019 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a78c45-0ade-43a7-bb91-4cbbb065df57 admin admin] PATH_INFO: `/v3/projects/689f614910a64bc78022bfb4505feba0` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:46.166134 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a78c45-0ade-43a7-bb91-4cbbb065df57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=689f614910a64bc78022bfb4505feba0)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:46.168130 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a78c45-0ade-43a7-bb91-4cbbb065df57 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:46.185660 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 853/1720] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:06:45 2022] POST /identity/v3/users => generated 279 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:46.193355 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced33c9-245d-4b7c-b66d-ebd7fc8dcedb None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:46.193904 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced33c9-245d-4b7c-b66d-ebd7fc8dcedb None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:46.196012 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced33c9-245d-4b7c-b66d-ebd7fc8dcedb None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:46.204531 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 854/1721] 10.222.0.27 () {54 vars in 912 bytes} [Sun Sep 4 01:06:46 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:06:46.206970 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced33c9-245d-4b7c-b66d-ebd7fc8dcedb None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:46.253732 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced33c9-245d-4b7c-b66d-ebd7fc8dcedb None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:46.267355 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e17671-98f8-4763-844c-7fabf28a5610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:46.268546 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e17671-98f8-4763-844c-7fabf28a5610 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:46.268833 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e17671-98f8-4763-844c-7fabf28a5610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:46.269103 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e17671-98f8-4763-844c-7fabf28a5610 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:46.269754 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e17671-98f8-4763-844c-7fabf28a5610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:46.271389 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e17671-98f8-4763-844c-7fabf28a5610 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:46.301330 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 855/1722] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:46.302721 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0cf7e4-4e2b-47eb-a591-fe7ae39d6be8 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:46.303013 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0cf7e4-4e2b-47eb-a591-fe7ae39d6be8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:46.303286 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0cf7e4-4e2b-47eb-a591-fe7ae39d6be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:46.345126 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-ae0cf7e4-4e2b-47eb-a591-fe7ae39d6be8 None None] MFA Rules not processed for user `861c278f8bf34b3bb407b2e051119ee9`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:46.349388 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ae0cf7e4-4e2b-47eb-a591-fe7ae39d6be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:46.366112 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-77a78c45-0ade-43a7-bb91-4cbbb065df57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '689f614910a64bc78022bfb4505feba0', 'request_id': 'req-77a78c45-0ade-43a7-bb91-4cbbb065df57'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:46.370751 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:46.478274 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 856/1723] 10.222.0.27 () {58 vars in 1009 bytes} [Sun Sep 4 01:06:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:46.478899 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0cf7e4-4e2b-47eb-a591-fe7ae39d6be8 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:46.497315 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-77a78c45-0ade-43a7-bb91-4cbbb065df57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '689f614910a64bc78022bfb4505feba0', 'request_id': 'req-77a78c45-0ade-43a7-bb91-4cbbb065df57'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:46.506146 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-77a78c45-0ade-43a7-bb91-4cbbb065df57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '689f614910a64bc78022bfb4505feba0', 'request_id': 'req-77a78c45-0ade-43a7-bb91-4cbbb065df57'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:46.544816 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-77a78c45-0ade-43a7-bb91-4cbbb065df57 admin admin] The token cache is being invalidate because project 689f614910a64bc78022bfb4505feba0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:06:46.545143 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-77a78c45-0ade-43a7-bb91-4cbbb065df57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:46.551039 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-77a78c45-0ade-43a7-bb91-4cbbb065df57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:46.559377 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 868/1724] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:06:46 2022] DELETE /identity/v3/projects/689f614910a64bc78022bfb4505feba0 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:46.561256 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a78c45-0ade-43a7-bb91-4cbbb065df57 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:46.568796 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-77a78c45-0ade-43a7-bb91-4cbbb065df57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:46.599543 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0cf7e4-4e2b-47eb-a591-fe7ae39d6be8 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:46.608806 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a78c45-0ade-43a7-bb91-4cbbb065df57 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:46.612295 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe624c37-e59a-48b7-a2ce-1a197533dbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:46.613041 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe624c37-e59a-48b7-a2ce-1a197533dbde admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:46.613707 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe624c37-e59a-48b7-a2ce-1a197533dbde admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:46.613707 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe624c37-e59a-48b7-a2ce-1a197533dbde admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:46.614360 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe624c37-e59a-48b7-a2ce-1a197533dbde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:46.616175 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe624c37-e59a-48b7-a2ce-1a197533dbde admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:46.618246 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af385b-ab85-4497-b0ee-50d3236eed92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:46.618878 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af385b-ab85-4497-b0ee-50d3236eed92 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:46.619164 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af385b-ab85-4497-b0ee-50d3236eed92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:46.619442 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af385b-ab85-4497-b0ee-50d3236eed92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:46.626665 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-08af385b-ab85-4497-b0ee-50d3236eed92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:46.630713 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 857/1725] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:46.630713 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe624c37-e59a-48b7-a2ce-1a197533dbde admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:46.632551 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-fe624c37-e59a-48b7-a2ce-1a197533dbde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:46.644804 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08af385b-ab85-4497-b0ee-50d3236eed92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:46.646332 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08af385b-ab85-4497-b0ee-50d3236eed92 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:46.647326 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe624c37-e59a-48b7-a2ce-1a197533dbde admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:46.656749 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 869/1726] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:06:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:06:46.658214 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af385b-ab85-4497-b0ee-50d3236eed92 service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:46.660526 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-08af385b-ab85-4497-b0ee-50d3236eed92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:46.670391 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e8dc8-7b56-4768-b01e-5cba7f4b727d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:46.671183 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e8dc8-7b56-4768-b01e-5cba7f4b727d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:46.671512 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e8dc8-7b56-4768-b01e-5cba7f4b727d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:46.672877 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e8dc8-7b56-4768-b01e-5cba7f4b727d admin admin] PATH_INFO: `/v3/users/7cf4744fca984e37a2731b29b5d65227` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:46.677954 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af385b-ab85-4497-b0ee-50d3236eed92 service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:46.682820 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0579c254-6f9b-482b-b6c3-a77a6da410a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:46.683640 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579c254-6f9b-482b-b6c3-a77a6da410a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:46.684433 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579c254-6f9b-482b-b6c3-a77a6da410a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:46.684433 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579c254-6f9b-482b-b6c3-a77a6da410a1 admin admin] PATH_INFO: `/v3/projects/f2c413bb59fd4322a7c493ecbad719bd/users/c7c14f3183254b19aaf33b966698567a/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:46.684900 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e8dc8-7b56-4768-b01e-5cba7f4b727d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cf4744fca984e37a2731b29b5d65227)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:46.686842 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e8dc8-7b56-4768-b01e-5cba7f4b727d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:46.724182 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0579c254-6f9b-482b-b6c3-a77a6da410a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2c413bb59fd4322a7c493ecbad719bd, user_id=c7c14f3183254b19aaf33b966698567a, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:46.729601 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0579c254-6f9b-482b-b6c3-a77a6da410a1 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:46.854679 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 870/1727] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:06:46 2022] PUT /identity/v3/projects/f2c413bb59fd4322a7c493ecbad719bd/users/c7c14f3183254b19aaf33b966698567a/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:46.866225 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace41d6-ceef-49fe-b16c-3f6ea7102843 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:46.866684 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace41d6-ceef-49fe-b16c-3f6ea7102843 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:46.867099 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace41d6-ceef-49fe-b16c-3f6ea7102843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:46.980745 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9e1e8dc8-7b56-4768-b01e-5cba7f4b727d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cf4744fca984e37a2731b29b5d65227'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:46.981560 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9e1e8dc8-7b56-4768-b01e-5cba7f4b727d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cf4744fca984e37a2731b29b5d65227'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:46.995722 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-5ace41d6-ceef-49fe-b16c-3f6ea7102843 None None] MFA Rules not processed for user `c7c14f3183254b19aaf33b966698567a`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:46.997826 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5ace41d6-ceef-49fe-b16c-3f6ea7102843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:46.999859 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9e1e8dc8-7b56-4768-b01e-5cba7f4b727d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cf4744fca984e37a2731b29b5d65227'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:47.106218 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9e1e8dc8-7b56-4768-b01e-5cba7f4b727d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cf4744fca984e37a2731b29b5d65227'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:47.107036 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9e1e8dc8-7b56-4768-b01e-5cba7f4b727d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cf4744fca984e37a2731b29b5d65227'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:47.138872 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:47.168035 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 871/1728] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:46 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:47.172278 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:47.185952 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 858/1729] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:06:46 2022] DELETE /identity/v3/users/7cf4744fca984e37a2731b29b5d65227 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:47.186731 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace41d6-ceef-49fe-b16c-3f6ea7102843 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:47.187745 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e8dc8-7b56-4768-b01e-5cba7f4b727d admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:47.190495 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5ace41d6-ceef-49fe-b16c-3f6ea7102843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:47.191955 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9e1e8dc8-7b56-4768-b01e-5cba7f4b727d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:47.409054 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e8dc8-7b56-4768-b01e-5cba7f4b727d admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:47.415530 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a13cde-e02e-4a9a-890c-283da8435262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:47.416414 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a13cde-e02e-4a9a-890c-283da8435262 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:47.416910 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a13cde-e02e-4a9a-890c-283da8435262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:47.417326 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a13cde-e02e-4a9a-890c-283da8435262 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:47.418169 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a13cde-e02e-4a9a-890c-283da8435262 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:47.420788 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a13cde-e02e-4a9a-890c-283da8435262 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:47.421885 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace41d6-ceef-49fe-b16c-3f6ea7102843 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:47.615894 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 859/1730] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:06:47 2022] POST /identity/v3/projects => generated 355 bytes in 429 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:47.617427 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a13cde-e02e-4a9a-890c-283da8435262 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:47.620412 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a70b018-91e6-409a-94d4-d1dea7ecd8a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:47.621609 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a70b018-91e6-409a-94d4-d1dea7ecd8a3 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:47.621852 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a70b018-91e6-409a-94d4-d1dea7ecd8a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:47.622125 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a70b018-91e6-409a-94d4-d1dea7ecd8a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:47.622880 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-14a13cde-e02e-4a9a-890c-283da8435262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:47.624316 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8a70b018-91e6-409a-94d4-d1dea7ecd8a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:47.765380 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a13cde-e02e-4a9a-890c-283da8435262 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:47.771710 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c4d5dd-3792-461f-9501-bdc80484b05f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '018802a8d9c041119ad708430f885a89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:47.779348 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4d5dd-3792-461f-9501-bdc80484b05f service placement] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:47.780043 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4d5dd-3792-461f-9501-bdc80484b05f service placement] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:47.780405 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4d5dd-3792-461f-9501-bdc80484b05f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:47.783906 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d7c4d5dd-3792-461f-9501-bdc80484b05f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:47.801011 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c4d5dd-3792-461f-9501-bdc80484b05f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:47.803306 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c4d5dd-3792-461f-9501-bdc80484b05f service placement] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:47.818345 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 860/1731] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:06:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:06:47.824503 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c4d5dd-3792-461f-9501-bdc80484b05f service placement] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:47.826436 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a70b018-91e6-409a-94d4-d1dea7ecd8a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:47.827579 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a70b018-91e6-409a-94d4-d1dea7ecd8a3 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:47.858644 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 872/1732] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:06:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 689 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:06:48.079234 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c4d5dd-3792-461f-9501-bdc80484b05f service placement] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:48.103734 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a46234-47bb-4e3f-bb2f-2fc9a3055a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:48.105446 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a46234-47bb-4e3f-bb2f-2fc9a3055a2a admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:48.105864 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a46234-47bb-4e3f-bb2f-2fc9a3055a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:48.106245 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a46234-47bb-4e3f-bb2f-2fc9a3055a2a admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:48.107046 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a46234-47bb-4e3f-bb2f-2fc9a3055a2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:48.110709 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a46234-47bb-4e3f-bb2f-2fc9a3055a2a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:48.248398 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82880c97-1512-4762-a82f-0ee2cd787e15 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:48.248922 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82880c97-1512-4762-a82f-0ee2cd787e15 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:48.249403 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82880c97-1512-4762-a82f-0ee2cd787e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:48.267727 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 861/1733] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:06:47 2022] POST /identity/v3/users => generated 284 bytes in 449 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:48.269083 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a46234-47bb-4e3f-bb2f-2fc9a3055a2a admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:48.286719 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a46234-47bb-4e3f-bb2f-2fc9a3055a2a admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:48.301617 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02352fd0-3d85-4b36-b646-527e130c0116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:48.303606 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02352fd0-3d85-4b36-b646-527e130c0116 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:48.304039 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02352fd0-3d85-4b36-b646-527e130c0116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:48.304807 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02352fd0-3d85-4b36-b646-527e130c0116 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:48.306019 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02352fd0-3d85-4b36-b646-527e130c0116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:48.308226 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02352fd0-3d85-4b36-b646-527e130c0116 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:48.345281 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-82880c97-1512-4762-a82f-0ee2cd787e15 None None] MFA Rules not processed for user `bd1c41fcdc724a128117a5288b8559f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:48.360427 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-82880c97-1512-4762-a82f-0ee2cd787e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:48.371500 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 862/1734] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:48.373058 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb93298-1e4d-4f64-a37d-6e35b439e34a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:48.382037 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb93298-1e4d-4f64-a37d-6e35b439e34a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:48.383572 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb93298-1e4d-4f64-a37d-6e35b439e34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:48.540267 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-fcb93298-1e4d-4f64-a37d-6e35b439e34a None None] MFA Rules not processed for user `9bef8dc1da4c44dbbafcaf3c813b3262`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:48.540267 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-fcb93298-1e4d-4f64-a37d-6e35b439e34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:48.606801 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 873/1735] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:48 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:06:48.608149 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82880c97-1512-4762-a82f-0ee2cd787e15 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:48.637740 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82880c97-1512-4762-a82f-0ee2cd787e15 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:48.652388 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ffb6b8-a455-4ad2-89f7-df4a333fb03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:48.653111 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ffb6b8-a455-4ad2-89f7-df4a333fb03d admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:48.653399 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ffb6b8-a455-4ad2-89f7-df4a333fb03d admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:48.653688 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ffb6b8-a455-4ad2-89f7-df4a333fb03d admin admin] PATH_INFO: `/v3/projects/e5adef07772a42e9a4fdb7800aaee4c9/users/f64a51e769884b5dba887808070d2848/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:48.694596 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ffb6b8-a455-4ad2-89f7-df4a333fb03d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5adef07772a42e9a4fdb7800aaee4c9, user_id=f64a51e769884b5dba887808070d2848, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:48.697432 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ffb6b8-a455-4ad2-89f7-df4a333fb03d admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:48.702675 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:48.824618 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 874/1736] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:06:48 2022] PUT /identity/v3/projects/e5adef07772a42e9a4fdb7800aaee4c9/users/f64a51e769884b5dba887808070d2848/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:48.827220 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ffb6b8-a455-4ad2-89f7-df4a333fb03d admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:48.859920 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 863/1737] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:48 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 483 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:49.012149 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ffb6b8-a455-4ad2-89f7-df4a333fb03d admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:49.013856 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb93298-1e4d-4f64-a37d-6e35b439e34a None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:49.015545 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-fcb93298-1e4d-4f64-a37d-6e35b439e34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:49.023450 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a18e06-ab42-4a1f-a818-1b4d37de6e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:49.025047 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a18e06-ab42-4a1f-a818-1b4d37de6e60 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:49.027915 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a18e06-ab42-4a1f-a818-1b4d37de6e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:49.028493 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a18e06-ab42-4a1f-a818-1b4d37de6e60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:49.029397 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a18e06-ab42-4a1f-a818-1b4d37de6e60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:49.053123 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a18e06-ab42-4a1f-a818-1b4d37de6e60 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:49.053346 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb93298-1e4d-4f64-a37d-6e35b439e34a None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:49.065080 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbe42d-da7f-416d-bf24-06e93e53c65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:49.065708 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbe42d-da7f-416d-bf24-06e93e53c65a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:49.065993 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbe42d-da7f-416d-bf24-06e93e53c65a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:49.066268 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbe42d-da7f-416d-bf24-06e93e53c65a admin admin] PATH_INFO: `/v3/users/51867bd28837452a944628ceefc075f8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:49.069218 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bbe42d-da7f-416d-bf24-06e93e53c65a admin admin] RBAC: Authorizing `identity:delete_user(user_id=51867bd28837452a944628ceefc075f8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:49.073232 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bbe42d-da7f-416d-bf24-06e93e53c65a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:49.074750 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 875/1738] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 01:06:49.087052 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a18e06-ab42-4a1f-a818-1b4d37de6e60 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:49.104775 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a18e06-ab42-4a1f-a818-1b4d37de6e60 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:49.112998 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62655590-a176-4ec7-865c-13dbccbda87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:49.114039 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62655590-a176-4ec7-865c-13dbccbda87d admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:49.114465 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62655590-a176-4ec7-865c-13dbccbda87d admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:49.114884 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62655590-a176-4ec7-865c-13dbccbda87d admin admin] PATH_INFO: `/v3/projects/e5adef07772a42e9a4fdb7800aaee4c9/users/f64a51e769884b5dba887808070d2848/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:49.120250 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62655590-a176-4ec7-865c-13dbccbda87d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5adef07772a42e9a4fdb7800aaee4c9, user_id=f64a51e769884b5dba887808070d2848, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:49.125529 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62655590-a176-4ec7-865c-13dbccbda87d admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:49.156403 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 876/1739] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:06:49 2022] PUT /identity/v3/projects/e5adef07772a42e9a4fdb7800aaee4c9/users/f64a51e769884b5dba887808070d2848/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:49.161985 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c742824-0f6b-4858-8a3c-b70dfb45d879 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:49.166147 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c742824-0f6b-4858-8a3c-b70dfb45d879 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:49.166147 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c742824-0f6b-4858-8a3c-b70dfb45d879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:49.257640 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-3c742824-0f6b-4858-8a3c-b70dfb45d879 None None] MFA Rules not processed for user `f64a51e769884b5dba887808070d2848`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:49.259872 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3c742824-0f6b-4858-8a3c-b70dfb45d879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:49.373663 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-27bbe42d-da7f-416d-bf24-06e93e53c65a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51867bd28837452a944628ceefc075f8'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:49.375039 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-27bbe42d-da7f-416d-bf24-06e93e53c65a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51867bd28837452a944628ceefc075f8'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:49.384820 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-27bbe42d-da7f-416d-bf24-06e93e53c65a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51867bd28837452a944628ceefc075f8'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:49.412436 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 877/1740] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:49 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:49.414730 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c742824-0f6b-4858-8a3c-b70dfb45d879 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:49.422893 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3c742824-0f6b-4858-8a3c-b70dfb45d879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:49.512678 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-27bbe42d-da7f-416d-bf24-06e93e53c65a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51867bd28837452a944628ceefc075f8'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:49.513686 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-27bbe42d-da7f-416d-bf24-06e93e53c65a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51867bd28837452a944628ceefc075f8'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:49.589336 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 864/1741] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:06:49 2022] DELETE /identity/v3/users/51867bd28837452a944628ceefc075f8 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:49.589584 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c742824-0f6b-4858-8a3c-b70dfb45d879 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:49.589788 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbe42d-da7f-416d-bf24-06e93e53c65a admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:49.589788 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-27bbe42d-da7f-416d-bf24-06e93e53c65a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:49.695401 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:49.708005 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bbb794-61c1-4cf2-bddb-0f5cbad0eeed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:49.708844 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbb794-61c1-4cf2-bddb-0f5cbad0eeed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:49.709131 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbb794-61c1-4cf2-bddb-0f5cbad0eeed admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:49.709413 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbb794-61c1-4cf2-bddb-0f5cbad0eeed admin admin] PATH_INFO: `/v3/projects/73ecaffec16d4a259949066dcd42b36c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:49.718883 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bbb794-61c1-4cf2-bddb-0f5cbad0eeed admin admin] RBAC: Authorizing `identity:delete_project(project_id=73ecaffec16d4a259949066dcd42b36c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:49.721920 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bbb794-61c1-4cf2-bddb-0f5cbad0eeed admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:49.751238 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbe42d-da7f-416d-bf24-06e93e53c65a admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:49.761848 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f170bf4-a541-4105-8636-6a27caf58235 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:49.766199 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f170bf4-a541-4105-8636-6a27caf58235 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:49.766199 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f170bf4-a541-4105-8636-6a27caf58235 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:49.766199 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f170bf4-a541-4105-8636-6a27caf58235 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:49.771956 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8f170bf4-a541-4105-8636-6a27caf58235 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:49.886854 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:49.912727 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f170bf4-a541-4105-8636-6a27caf58235 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:49.914481 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f170bf4-a541-4105-8636-6a27caf58235 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:49.943989 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a9bbb794-61c1-4cf2-bddb-0f5cbad0eeed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73ecaffec16d4a259949066dcd42b36c', 'request_id': 'req-a9bbb794-61c1-4cf2-bddb-0f5cbad0eeed'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:49.978020 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 865/1742] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:06:49 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:06:49.982711 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604a087-aa7a-4071-a45e-565c45ab736b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:49.983087 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604a087-aa7a-4071-a45e-565c45ab736b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:49.985620 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604a087-aa7a-4071-a45e-565c45ab736b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:50.091645 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-7604a087-aa7a-4071-a45e-565c45ab736b None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:50.093012 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7604a087-aa7a-4071-a45e-565c45ab736b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:50.097758 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a9bbb794-61c1-4cf2-bddb-0f5cbad0eeed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73ecaffec16d4a259949066dcd42b36c', 'request_id': 'req-a9bbb794-61c1-4cf2-bddb-0f5cbad0eeed'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:50.110296 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a9bbb794-61c1-4cf2-bddb-0f5cbad0eeed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73ecaffec16d4a259949066dcd42b36c', 'request_id': 'req-a9bbb794-61c1-4cf2-bddb-0f5cbad0eeed'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:50.126510 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a9bbb794-61c1-4cf2-bddb-0f5cbad0eeed admin admin] The token cache is being invalidate because project 73ecaffec16d4a259949066dcd42b36c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:06:50.126903 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a9bbb794-61c1-4cf2-bddb-0f5cbad0eeed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:50.132183 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a9bbb794-61c1-4cf2-bddb-0f5cbad0eeed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:50.137791 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 878/1743] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:06:49 2022] DELETE /identity/v3/projects/73ecaffec16d4a259949066dcd42b36c => generated 0 bytes in 722 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:50.140061 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05705c3e-9057-4868-bef7-49f23234fda6 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:50.140416 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05705c3e-9057-4868-bef7-49f23234fda6 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:50.140795 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05705c3e-9057-4868-bef7-49f23234fda6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:50.188283 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-05705c3e-9057-4868-bef7-49f23234fda6 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:50.190502 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-05705c3e-9057-4868-bef7-49f23234fda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:50.251650 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 866/1744] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:50.254499 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa67a14-7dea-4166-b187-ee915741dcfc None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:50.255582 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa67a14-7dea-4166-b187-ee915741dcfc None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:50.256640 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa67a14-7dea-4166-b187-ee915741dcfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:50.288640 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-4fa67a14-7dea-4166-b187-ee915741dcfc None None] MFA Rules not processed for user `0c5e9731f6204b44841b8524762d5a77`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:50.291471 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4fa67a14-7dea-4166-b187-ee915741dcfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:50.324637 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 879/1745] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:06:50.325681 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05705c3e-9057-4868-bef7-49f23234fda6 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:50.334144 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-05705c3e-9057-4868-bef7-49f23234fda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:50.430678 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 867/1746] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:50.431613 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa67a14-7dea-4166-b187-ee915741dcfc None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:50.437697 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4fa67a14-7dea-4166-b187-ee915741dcfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:50.484380 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05705c3e-9057-4868-bef7-49f23234fda6 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:50.493580 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b07d09-4a14-4f0c-a0d2-e230edead97d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:50.500642 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b07d09-4a14-4f0c-a0d2-e230edead97d service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:50.500642 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b07d09-4a14-4f0c-a0d2-e230edead97d service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:50.500642 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b07d09-4a14-4f0c-a0d2-e230edead97d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:50.522721 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b07d09-4a14-4f0c-a0d2-e230edead97d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:50.522721 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b07d09-4a14-4f0c-a0d2-e230edead97d service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:50.545366 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 880/1747] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:06:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:06:50.546840 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97fd1fe-213b-43b2-bded-e5814a45beda None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:50.549979 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97fd1fe-213b-43b2-bded-e5814a45beda None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:50.550433 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97fd1fe-213b-43b2-bded-e5814a45beda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:50.555352 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa67a14-7dea-4166-b187-ee915741dcfc None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:50.564868 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0011b-1c38-46bb-a404-21583db828b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:50.566004 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0011b-1c38-46bb-a404-21583db828b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:50.566294 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0011b-1c38-46bb-a404-21583db828b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:50.566574 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0011b-1c38-46bb-a404-21583db828b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:50.573194 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b9f0011b-1c38-46bb-a404-21583db828b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:50.602544 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-a97fd1fe-213b-43b2-bded-e5814a45beda None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:50.603965 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a97fd1fe-213b-43b2-bded-e5814a45beda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:50.628325 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 881/1748] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:50.631126 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97fd1fe-213b-43b2-bded-e5814a45beda None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:50.648128 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97fd1fe-213b-43b2-bded-e5814a45beda None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:50.664678 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3116db-044e-42b7-bdab-c746dd7364d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:50.665421 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3116db-044e-42b7-bdab-c746dd7364d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:50.667336 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3116db-044e-42b7-bdab-c746dd7364d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:50.668581 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3116db-044e-42b7-bdab-c746dd7364d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:50.669626 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3116db-044e-42b7-bdab-c746dd7364d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:06:50.671371 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3116db-044e-42b7-bdab-c746dd7364d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:50.676285 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3116db-044e-42b7-bdab-c746dd7364d8 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:50.691211 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 882/1749] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:06:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:06:50.696609 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3116db-044e-42b7-bdab-c746dd7364d8 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:50.718292 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3116db-044e-42b7-bdab-c746dd7364d8 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:50.724006 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f0011b-1c38-46bb-a404-21583db828b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:50.726099 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f0011b-1c38-46bb-a404-21583db828b9 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:50.726519 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c44e9-04d2-4e87-b1bb-66c258ae948a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:50.727223 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c44e9-04d2-4e87-b1bb-66c258ae948a admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:50.727574 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c44e9-04d2-4e87-b1bb-66c258ae948a admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:50.727994 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c44e9-04d2-4e87-b1bb-66c258ae948a admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:50.728818 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182c44e9-04d2-4e87-b1bb-66c258ae948a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:50.731309 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182c44e9-04d2-4e87-b1bb-66c258ae948a admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:50.738320 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 868/1750] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:06:50 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:06:50.890759 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 883/1751] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:06:50 2022] POST /identity/v3/projects => generated 387 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:50.895337 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0011b-1c38-46bb-a404-21583db828b9 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:50.917186 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0011b-1c38-46bb-a404-21583db828b9 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:50.990562 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240695cd-4912-4ecb-aa40-7ac67829df18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:50.991706 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240695cd-4912-4ecb-aa40-7ac67829df18 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:51.006889 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240695cd-4912-4ecb-aa40-7ac67829df18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:51.007506 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240695cd-4912-4ecb-aa40-7ac67829df18 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:51.018744 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240695cd-4912-4ecb-aa40-7ac67829df18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:51.027327 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240695cd-4912-4ecb-aa40-7ac67829df18 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:51.188852 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c44e9-04d2-4e87-b1bb-66c258ae948a admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:51.191156 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 869/1752] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:06:50 2022] POST /identity/v3/users => generated 294 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:51.207650 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c44e9-04d2-4e87-b1bb-66c258ae948a admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:51.219195 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679363f-9cef-4218-9dae-528dacf07660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:51.220186 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679363f-9cef-4218-9dae-528dacf07660 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:51.221243 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679363f-9cef-4218-9dae-528dacf07660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:51.226867 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679363f-9cef-4218-9dae-528dacf07660 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:51.226867 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679363f-9cef-4218-9dae-528dacf07660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:51.228363 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679363f-9cef-4218-9dae-528dacf07660 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:51.247477 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 884/1753] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:51.251530 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240695cd-4912-4ecb-aa40-7ac67829df18 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:51.264407 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240695cd-4912-4ecb-aa40-7ac67829df18 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:51.287490 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a354c50-2625-43ea-aa45-e8ea0207fd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:51.288027 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a354c50-2625-43ea-aa45-e8ea0207fd4d admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:51.288307 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a354c50-2625-43ea-aa45-e8ea0207fd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:51.288622 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a354c50-2625-43ea-aa45-e8ea0207fd4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:51.289280 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a354c50-2625-43ea-aa45-e8ea0207fd4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:51.290890 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a354c50-2625-43ea-aa45-e8ea0207fd4d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:51.344402 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679363f-9cef-4218-9dae-528dacf07660 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:51.349213 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 870/1754] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:51.359380 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679363f-9cef-4218-9dae-528dacf07660 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:51.383331 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff5a8e-bdf0-447b-aaea-3434ef65ced7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:51.384049 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff5a8e-bdf0-447b-aaea-3434ef65ced7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:51.384333 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff5a8e-bdf0-447b-aaea-3434ef65ced7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:51.384807 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff5a8e-bdf0-447b-aaea-3434ef65ced7 admin admin] PATH_INFO: `/v3/projects/aac24051b0784e59ac1c0111f4f579d1/users/4d495385ebe543f7b57ee1c1519f4f21/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:51.409979 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff5a8e-bdf0-447b-aaea-3434ef65ced7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aac24051b0784e59ac1c0111f4f579d1, user_id=4d495385ebe543f7b57ee1c1519f4f21, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:51.413610 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff5a8e-bdf0-447b-aaea-3434ef65ced7 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:51.499010 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 885/1755] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:06:51 2022] PUT /identity/v3/projects/aac24051b0784e59ac1c0111f4f579d1/users/4d495385ebe543f7b57ee1c1519f4f21/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:51.514612 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc7a59-d910-49ba-a0bd-34041d4e358f None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:51.515115 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc7a59-d910-49ba-a0bd-34041d4e358f None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:51.515389 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc7a59-d910-49ba-a0bd-34041d4e358f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:51.556041 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff5a8e-bdf0-447b-aaea-3434ef65ced7 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:51.559559 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b0ff5a8e-bdf0-447b-aaea-3434ef65ced7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:51.586409 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-15bc7a59-d910-49ba-a0bd-34041d4e358f None None] MFA Rules not processed for user `4d495385ebe543f7b57ee1c1519f4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:51.599943 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-15bc7a59-d910-49ba-a0bd-34041d4e358f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:51.759304 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff5a8e-bdf0-447b-aaea-3434ef65ced7 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:51.774245 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed673c6f-2312-4008-835b-9b353bf5fcf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:51.774839 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed673c6f-2312-4008-835b-9b353bf5fcf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:51.775129 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed673c6f-2312-4008-835b-9b353bf5fcf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:51.775408 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed673c6f-2312-4008-835b-9b353bf5fcf7 admin admin] PATH_INFO: `/v3/projects/b087ee3b57aa43fa8f34151318e982d6` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:51.779829 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed673c6f-2312-4008-835b-9b353bf5fcf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b087ee3b57aa43fa8f34151318e982d6)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:51.783515 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed673c6f-2312-4008-835b-9b353bf5fcf7 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:51.791782 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 871/1756] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:51.797559 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc7a59-d910-49ba-a0bd-34041d4e358f None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:51.842050 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc7a59-d910-49ba-a0bd-34041d4e358f None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:51.872620 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907fa81-2515-4d3c-ac65-99fda0c3de4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:51.873324 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907fa81-2515-4d3c-ac65-99fda0c3de4e admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:51.876647 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907fa81-2515-4d3c-ac65-99fda0c3de4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:51.879164 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907fa81-2515-4d3c-ac65-99fda0c3de4e admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:51.879880 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1907fa81-2515-4d3c-ac65-99fda0c3de4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:51.888239 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1907fa81-2515-4d3c-ac65-99fda0c3de4e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:51.926412 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:51.995287 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 872/1757] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:06:51 2022] POST /identity/v3/projects => generated 385 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:51.999129 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907fa81-2515-4d3c-ac65-99fda0c3de4e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:52.006425 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ed673c6f-2312-4008-835b-9b353bf5fcf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b087ee3b57aa43fa8f34151318e982d6', 'request_id': 'req-ed673c6f-2312-4008-835b-9b353bf5fcf7'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:52.084502 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ed673c6f-2312-4008-835b-9b353bf5fcf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b087ee3b57aa43fa8f34151318e982d6', 'request_id': 'req-ed673c6f-2312-4008-835b-9b353bf5fcf7'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:52.091769 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ed673c6f-2312-4008-835b-9b353bf5fcf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b087ee3b57aa43fa8f34151318e982d6', 'request_id': 'req-ed673c6f-2312-4008-835b-9b353bf5fcf7'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:52.125441 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907fa81-2515-4d3c-ac65-99fda0c3de4e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:52.129705 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ed673c6f-2312-4008-835b-9b353bf5fcf7 admin admin] The token cache is being invalidate because project b087ee3b57aa43fa8f34151318e982d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:06:52.130964 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ed673c6f-2312-4008-835b-9b353bf5fcf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:52.132140 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b97a3-9c2d-41b8-b6e1-65acee9fd9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:52.133847 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ed673c6f-2312-4008-835b-9b353bf5fcf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:52.137171 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 886/1758] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:06:51 2022] DELETE /identity/v3/projects/b087ee3b57aa43fa8f34151318e982d6 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:52.142472 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed673c6f-2312-4008-835b-9b353bf5fcf7 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:52.144036 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b97a3-9c2d-41b8-b6e1-65acee9fd9b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:52.144501 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b97a3-9c2d-41b8-b6e1-65acee9fd9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:52.145101 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b97a3-9c2d-41b8-b6e1-65acee9fd9b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:52.145806 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71b97a3-9c2d-41b8-b6e1-65acee9fd9b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:52.150447 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71b97a3-9c2d-41b8-b6e1-65acee9fd9b3 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:52.156644 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ed673c6f-2312-4008-835b-9b353bf5fcf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:52.311707 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 873/1759] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:06:51 2022] POST /identity/v3/users => generated 299 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:52.314293 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b97a3-9c2d-41b8-b6e1-65acee9fd9b3 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:52.326623 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d71b97a3-9c2d-41b8-b6e1-65acee9fd9b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:52.432178 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b97a3-9c2d-41b8-b6e1-65acee9fd9b3 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:52.470174 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8e649-4837-4a3c-8aca-706e46b78658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:52.471719 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8e649-4837-4a3c-8aca-706e46b78658 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:52.472180 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8e649-4837-4a3c-8aca-706e46b78658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:52.494251 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed673c6f-2312-4008-835b-9b353bf5fcf7 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:52.500018 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8e649-4837-4a3c-8aca-706e46b78658 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:52.507688 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a8e649-4837-4a3c-8aca-706e46b78658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:52.512330 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a8e649-4837-4a3c-8aca-706e46b78658 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:52.518617 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc65fc9d-8939-481d-8193-5a92aa5e0a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:52.520547 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65fc9d-8939-481d-8193-5a92aa5e0a8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:52.521054 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65fc9d-8939-481d-8193-5a92aa5e0a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:52.521534 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65fc9d-8939-481d-8193-5a92aa5e0a8d admin admin] PATH_INFO: `/v3/users/33d4d922264d4446b55e789cda2f5df0` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:52.523036 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc65fc9d-8939-481d-8193-5a92aa5e0a8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=33d4d922264d4446b55e789cda2f5df0)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:52.526369 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc65fc9d-8939-481d-8193-5a92aa5e0a8d admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:52.581436 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 874/1760] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:52.584321 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8e649-4837-4a3c-8aca-706e46b78658 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:52.610367 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8e649-4837-4a3c-8aca-706e46b78658 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:52.625076 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f79e3-936e-44f8-b084-60d3cb11da80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:52.626329 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f79e3-936e-44f8-b084-60d3cb11da80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:52.627814 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f79e3-936e-44f8-b084-60d3cb11da80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:52.628266 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f79e3-936e-44f8-b084-60d3cb11da80 admin admin] PATH_INFO: `/v3/projects/66780a4062684b0199c9c3ebceb9af8d/users/1f8b1011c2b24f3b85e8cfb28aea8f49/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:52.670701 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641f79e3-936e-44f8-b084-60d3cb11da80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66780a4062684b0199c9c3ebceb9af8d, user_id=1f8b1011c2b24f3b85e8cfb28aea8f49, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:52.674011 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641f79e3-936e-44f8-b084-60d3cb11da80 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:52.795049 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 875/1761] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:06:52 2022] PUT /identity/v3/projects/66780a4062684b0199c9c3ebceb9af8d/users/1f8b1011c2b24f3b85e8cfb28aea8f49/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:06:52.799536 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f79e3-936e-44f8-b084-60d3cb11da80 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:52.811325 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-dc65fc9d-8939-481d-8193-5a92aa5e0a8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33d4d922264d4446b55e789cda2f5df0'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:52.815071 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-dc65fc9d-8939-481d-8193-5a92aa5e0a8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33d4d922264d4446b55e789cda2f5df0'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:52.914807 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f79e3-936e-44f8-b084-60d3cb11da80 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:52.921264 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d7bcb-fed8-4e88-856c-4525ae1f8c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:52.927206 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d7bcb-fed8-4e88-856c-4525ae1f8c64 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:52.927206 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d7bcb-fed8-4e88-856c-4525ae1f8c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:52.927206 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d7bcb-fed8-4e88-856c-4525ae1f8c64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:52.927206 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876d7bcb-fed8-4e88-856c-4525ae1f8c64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:52.938166 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876d7bcb-fed8-4e88-856c-4525ae1f8c64 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:52.948877 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-dc65fc9d-8939-481d-8193-5a92aa5e0a8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33d4d922264d4446b55e789cda2f5df0'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:52.949488 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-dc65fc9d-8939-481d-8193-5a92aa5e0a8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33d4d922264d4446b55e789cda2f5df0'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:52.965386 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 876/1762] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:52.966317 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d7bcb-fed8-4e88-856c-4525ae1f8c64 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:52.967449 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-dc65fc9d-8939-481d-8193-5a92aa5e0a8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33d4d922264d4446b55e789cda2f5df0'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:52.968106 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-876d7bcb-fed8-4e88-856c-4525ae1f8c64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:52.985012 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d7bcb-fed8-4e88-856c-4525ae1f8c64 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:52.991812 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f872d7-b98e-4c14-96a6-eaea678bb238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:52.992580 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f872d7-b98e-4c14-96a6-eaea678bb238 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:52.992869 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f872d7-b98e-4c14-96a6-eaea678bb238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:52.993150 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f872d7-b98e-4c14-96a6-eaea678bb238 admin admin] PATH_INFO: `/v3/projects/66780a4062684b0199c9c3ebceb9af8d/users/1f8b1011c2b24f3b85e8cfb28aea8f49/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:52.994318 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 887/1763] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:06:52 2022] DELETE /identity/v3/users/33d4d922264d4446b55e789cda2f5df0 => generated 0 bytes in 853 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:52.995466 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f872d7-b98e-4c14-96a6-eaea678bb238 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66780a4062684b0199c9c3ebceb9af8d, user_id=1f8b1011c2b24f3b85e8cfb28aea8f49, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:52.998737 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f872d7-b98e-4c14-96a6-eaea678bb238 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:53.045592 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 877/1764] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:06:52 2022] PUT /identity/v3/projects/66780a4062684b0199c9c3ebceb9af8d/users/1f8b1011c2b24f3b85e8cfb28aea8f49/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:53.060370 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51b230-fe88-407c-99d8-039d445cf677 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:53.061438 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51b230-fe88-407c-99d8-039d445cf677 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:53.062046 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51b230-fe88-407c-99d8-039d445cf677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:53.127855 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-fa51b230-fe88-407c-99d8-039d445cf677 None None] MFA Rules not processed for user `1f8b1011c2b24f3b85e8cfb28aea8f49`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:53.129675 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-fa51b230-fe88-407c-99d8-039d445cf677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:53.172036 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:53.320601 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 888/1765] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:53 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:53.333424 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f872d7-b98e-4c14-96a6-eaea678bb238 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:53.336178 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-36f872d7-b98e-4c14-96a6-eaea678bb238 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:53.474156 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f872d7-b98e-4c14-96a6-eaea678bb238 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:53.481494 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad184f6a-9645-4c20-916e-b2bfb72fe96d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:53.483265 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad184f6a-9645-4c20-916e-b2bfb72fe96d service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:53.483402 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad184f6a-9645-4c20-916e-b2bfb72fe96d service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:53.483692 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad184f6a-9645-4c20-916e-b2bfb72fe96d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:53.498160 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad184f6a-9645-4c20-916e-b2bfb72fe96d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:53.500339 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad184f6a-9645-4c20-916e-b2bfb72fe96d service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:53.535390 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 878/1766] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:06:53 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:06:53.723089 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51b230-fe88-407c-99d8-039d445cf677 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:53.742701 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad184f6a-9645-4c20-916e-b2bfb72fe96d service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:53.742701 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ad184f6a-9645-4c20-916e-b2bfb72fe96d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:53.840213 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51b230-fe88-407c-99d8-039d445cf677 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:53.850675 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14413277-7535-4829-a7e9-b418919585ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:53.866374 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14413277-7535-4829-a7e9-b418919585ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:53.868152 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14413277-7535-4829-a7e9-b418919585ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:53.868574 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14413277-7535-4829-a7e9-b418919585ff admin admin] PATH_INFO: `/v3/users/4d495385ebe543f7b57ee1c1519f4f21` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:53.877890 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14413277-7535-4829-a7e9-b418919585ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d495385ebe543f7b57ee1c1519f4f21)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:53.890422 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14413277-7535-4829-a7e9-b418919585ff admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:53.935891 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad184f6a-9645-4c20-916e-b2bfb72fe96d service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:53.941794 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1350fc03-a855-49da-9a9e-5449cc4af61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:53.942569 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350fc03-a855-49da-9a9e-5449cc4af61f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:53.942957 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350fc03-a855-49da-9a9e-5449cc4af61f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:53.943334 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350fc03-a855-49da-9a9e-5449cc4af61f admin admin] PATH_INFO: `/v3/users/128e8929aef44054b4aa4e4dc7d13f20` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:53.944637 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1350fc03-a855-49da-9a9e-5449cc4af61f admin admin] RBAC: Authorizing `identity:delete_user(user_id=128e8929aef44054b4aa4e4dc7d13f20)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:53.947034 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1350fc03-a855-49da-9a9e-5449cc4af61f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:54.190792 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-14413277-7535-4829-a7e9-b418919585ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d495385ebe543f7b57ee1c1519f4f21'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:54.194622 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-14413277-7535-4829-a7e9-b418919585ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d495385ebe543f7b57ee1c1519f4f21'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:54.253802 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1350fc03-a855-49da-9a9e-5449cc4af61f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '128e8929aef44054b4aa4e4dc7d13f20'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:54.253802 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1350fc03-a855-49da-9a9e-5449cc4af61f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '128e8929aef44054b4aa4e4dc7d13f20'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:54.260665 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1350fc03-a855-49da-9a9e-5449cc4af61f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '128e8929aef44054b4aa4e4dc7d13f20'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:54.295030 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-14413277-7535-4829-a7e9-b418919585ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d495385ebe543f7b57ee1c1519f4f21'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:54.295959 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-14413277-7535-4829-a7e9-b418919585ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d495385ebe543f7b57ee1c1519f4f21'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:54.320245 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:54.330369 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-14413277-7535-4829-a7e9-b418919585ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d495385ebe543f7b57ee1c1519f4f21'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:54.394703 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 889/1767] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:06:53 2022] DELETE /identity/v3/users/4d495385ebe543f7b57ee1c1519f4f21 => generated 0 bytes in 673 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:54.397788 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14413277-7535-4829-a7e9-b418919585ff admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:54.398918 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-14413277-7535-4829-a7e9-b418919585ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:54.400970 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1350fc03-a855-49da-9a9e-5449cc4af61f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '128e8929aef44054b4aa4e4dc7d13f20'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:54.403020 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1350fc03-a855-49da-9a9e-5449cc4af61f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '128e8929aef44054b4aa4e4dc7d13f20'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:54.443233 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 879/1768] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:06:53 2022] DELETE /identity/v3/users/128e8929aef44054b4aa4e4dc7d13f20 => generated 0 bytes in 705 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:54.447624 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470c06e-8d15-4422-ada5-97567e4afc5c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:54.450292 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470c06e-8d15-4422-ada5-97567e4afc5c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:54.452053 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470c06e-8d15-4422-ada5-97567e4afc5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:54.498816 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-b470c06e-8d15-4422-ada5-97567e4afc5c None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:54.501223 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b470c06e-8d15-4422-ada5-97567e4afc5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:54.576776 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14413277-7535-4829-a7e9-b418919585ff admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:54.648932 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 880/1769] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:54.648932 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a8834-7056-4ac6-8bfc-364e0da53512 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:54.649163 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a8834-7056-4ac6-8bfc-364e0da53512 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:54.649545 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a8834-7056-4ac6-8bfc-364e0da53512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:54.693332 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-e01a8834-7056-4ac6-8bfc-364e0da53512 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:54.695297 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e01a8834-7056-4ac6-8bfc-364e0da53512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:54.697991 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcff62-bb62-4722-8955-39e222f6515f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:54.698719 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcff62-bb62-4722-8955-39e222f6515f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:54.699374 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcff62-bb62-4722-8955-39e222f6515f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:54.699374 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcff62-bb62-4722-8955-39e222f6515f admin admin] PATH_INFO: `/v3/projects/d4cdd34edf834802a9e9f83f90f38197` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:54.700680 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bcff62-bb62-4722-8955-39e222f6515f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4cdd34edf834802a9e9f83f90f38197)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:54.702917 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bcff62-bb62-4722-8955-39e222f6515f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:54.710057 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 881/1770] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:54.722373 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7c36b-38ca-4e84-af07-0a0931a699e0 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:54.722951 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7c36b-38ca-4e84-af07-0a0931a699e0 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:54.723318 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7c36b-38ca-4e84-af07-0a0931a699e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:54.804612 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-efb7c36b-38ca-4e84-af07-0a0931a699e0 None None] MFA Rules not processed for user `d6d01ca673a1404c9b4cea51036c595c`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:54.806040 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-efb7c36b-38ca-4e84-af07-0a0931a699e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:54.827615 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-d9bcff62-bb62-4722-8955-39e222f6515f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4cdd34edf834802a9e9f83f90f38197', 'request_id': 'req-d9bcff62-bb62-4722-8955-39e222f6515f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:54.881711 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 882/1771] 10.222.0.27 () {58 vars in 1009 bytes} [Sun Sep 4 01:06:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:54.883825 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb7c36b-38ca-4e84-af07-0a0931a699e0 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:54.885182 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-efb7c36b-38ca-4e84-af07-0a0931a699e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:54.946513 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-d9bcff62-bb62-4722-8955-39e222f6515f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4cdd34edf834802a9e9f83f90f38197', 'request_id': 'req-d9bcff62-bb62-4722-8955-39e222f6515f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:54.954714 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-d9bcff62-bb62-4722-8955-39e222f6515f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4cdd34edf834802a9e9f83f90f38197', 'request_id': 'req-d9bcff62-bb62-4722-8955-39e222f6515f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:54.995634 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-d9bcff62-bb62-4722-8955-39e222f6515f admin admin] The token cache is being invalidate because project d4cdd34edf834802a9e9f83f90f38197 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:06:54.996100 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-d9bcff62-bb62-4722-8955-39e222f6515f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:54.998224 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-d9bcff62-bb62-4722-8955-39e222f6515f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:55.003021 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 890/1772] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:06:54 2022] DELETE /identity/v3/projects/d4cdd34edf834802a9e9f83f90f38197 => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:55.004314 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcff62-bb62-4722-8955-39e222f6515f admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:55.007976 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb7c36b-38ca-4e84-af07-0a0931a699e0 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:55.027231 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b2edf-fb2b-45b8-95bb-fc6fbde22291 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:55.027975 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b2edf-fb2b-45b8-95bb-fc6fbde22291 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:55.028360 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b2edf-fb2b-45b8-95bb-fc6fbde22291 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:55.028750 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b2edf-fb2b-45b8-95bb-fc6fbde22291 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:55.033972 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2b5b2edf-fb2b-45b8-95bb-fc6fbde22291 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:55.047988 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d9bcff62-bb62-4722-8955-39e222f6515f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:55.080556 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcff62-bb62-4722-8955-39e222f6515f admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:55.090629 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef30ac34-a934-4053-bde5-42d4eb0e4850 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:55.091508 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30ac34-a934-4053-bde5-42d4eb0e4850 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:55.092012 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30ac34-a934-4053-bde5-42d4eb0e4850 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:55.093249 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30ac34-a934-4053-bde5-42d4eb0e4850 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:55.097971 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ef30ac34-a934-4053-bde5-42d4eb0e4850 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:55.315537 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5b2edf-fb2b-45b8-95bb-fc6fbde22291 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:55.320691 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5b2edf-fb2b-45b8-95bb-fc6fbde22291 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:55.346213 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef30ac34-a934-4053-bde5-42d4eb0e4850 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:55.350956 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef30ac34-a934-4053-bde5-42d4eb0e4850 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:55.353723 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 883/1773] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:06:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:06:55.357578 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be481326-eb35-4211-8833-c9eacfc31587 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:55.357860 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be481326-eb35-4211-8833-c9eacfc31587 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:55.358131 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be481326-eb35-4211-8833-c9eacfc31587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:55.375414 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 891/1774] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:06:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:06:55.391336 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-be481326-eb35-4211-8833-c9eacfc31587 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:55.392657 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-be481326-eb35-4211-8833-c9eacfc31587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:55.408017 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 884/1775] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:55.416303 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10fbee-8087-4593-b1ff-7a162a572d06 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:55.416620 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10fbee-8087-4593-b1ff-7a162a572d06 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:55.416984 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10fbee-8087-4593-b1ff-7a162a572d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:55.463418 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-6a10fbee-8087-4593-b1ff-7a162a572d06 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:55.466024 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-6a10fbee-8087-4593-b1ff-7a162a572d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:55.477933 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 892/1776] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:06:55.481321 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be481326-eb35-4211-8833-c9eacfc31587 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:55.493927 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be481326-eb35-4211-8833-c9eacfc31587 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:55.500894 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:55.505347 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc071fc-cefe-405d-843d-dbdf68c2a944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:55.506150 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc071fc-cefe-405d-843d-dbdf68c2a944 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:55.506582 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc071fc-cefe-405d-843d-dbdf68c2a944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:55.507413 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc071fc-cefe-405d-843d-dbdf68c2a944 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:55.508286 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc071fc-cefe-405d-843d-dbdf68c2a944 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:06:55.508641 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc071fc-cefe-405d-843d-dbdf68c2a944 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:55.510509 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc071fc-cefe-405d-843d-dbdf68c2a944 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:55.532756 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10fbee-8087-4593-b1ff-7a162a572d06 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:55.534876 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-6a10fbee-8087-4593-b1ff-7a162a572d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:55.555437 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 885/1777] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:06:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:06:55.556652 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc071fc-cefe-405d-843d-dbdf68c2a944 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:55.631036 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc071fc-cefe-405d-843d-dbdf68c2a944 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:55.656039 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a41c99-86c7-4ac9-ac8e-3569a89fa459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:55.660072 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a41c99-86c7-4ac9-ac8e-3569a89fa459 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:55.660072 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a41c99-86c7-4ac9-ac8e-3569a89fa459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:55.660072 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a41c99-86c7-4ac9-ac8e-3569a89fa459 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:55.660072 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a41c99-86c7-4ac9-ac8e-3569a89fa459 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:55.680555 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a41c99-86c7-4ac9-ac8e-3569a89fa459 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:55.833564 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 886/1778] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:06:55 2022] POST /identity/v3/projects => generated 383 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:55.838203 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a41c99-86c7-4ac9-ac8e-3569a89fa459 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:55.879544 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10fbee-8087-4593-b1ff-7a162a572d06 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:55.964009 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a41c99-86c7-4ac9-ac8e-3569a89fa459 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:56.005861 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd7e7ab-fedf-4b02-b834-8ac356cb8c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:56.008473 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd7e7ab-fedf-4b02-b834-8ac356cb8c26 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:56.009254 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd7e7ab-fedf-4b02-b834-8ac356cb8c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:56.011717 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd7e7ab-fedf-4b02-b834-8ac356cb8c26 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:56.012680 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd7e7ab-fedf-4b02-b834-8ac356cb8c26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:56.019451 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd7e7ab-fedf-4b02-b834-8ac356cb8c26 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:56.019874 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb90a19-8b77-4b40-a240-826f22f1c8d3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2297d8b3fe8e47eb9fd39a6e3eea8f54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:56.022668 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb90a19-8b77-4b40-a240-826f22f1c8d3 service swift] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:56.023117 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb90a19-8b77-4b40-a240-826f22f1c8d3 service swift] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:56.023498 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb90a19-8b77-4b40-a240-826f22f1c8d3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:56.030896 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2eb90a19-8b77-4b40-a240-826f22f1c8d3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:56.136642 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb90a19-8b77-4b40-a240-826f22f1c8d3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:56.138385 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb90a19-8b77-4b40-a240-826f22f1c8d3 service swift] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:56.141363 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 887/1779] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:06:55 2022] POST /identity/v3/users => generated 292 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:56.146854 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7326ba4-91d1-4742-977a-02f07736026e None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:56.146854 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7326ba4-91d1-4742-977a-02f07736026e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:56.146854 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7326ba4-91d1-4742-977a-02f07736026e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:56.151619 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 893/1780] 10.222.0.27 () {60 vars in 1384 bytes} [Sun Sep 4 01:06:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 04 01:06:56.152550 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb90a19-8b77-4b40-a240-826f22f1c8d3 service swift] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:56.170837 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb90a19-8b77-4b40-a240-826f22f1c8d3 service swift] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:56.175523 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76591708-bc62-410e-89d4-7cc0785418ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:56.175976 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76591708-bc62-410e-89d4-7cc0785418ed admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:56.176235 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76591708-bc62-410e-89d4-7cc0785418ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:56.183896 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76591708-bc62-410e-89d4-7cc0785418ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:56.184560 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76591708-bc62-410e-89d4-7cc0785418ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:56.186180 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76591708-bc62-410e-89d4-7cc0785418ed admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:56.207035 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 894/1781] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:56.207970 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76591708-bc62-410e-89d4-7cc0785418ed admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:56.213375 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-b7326ba4-91d1-4742-977a-02f07736026e None None] MFA Rules not processed for user `861c278f8bf34b3bb407b2e051119ee9`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:56.216849 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b7326ba4-91d1-4742-977a-02f07736026e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:56.249478 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76591708-bc62-410e-89d4-7cc0785418ed admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:56.276297 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e6f2f4-61b3-4252-8c4b-d2113d85d2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:56.277311 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6f2f4-61b3-4252-8c4b-d2113d85d2d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:56.278054 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6f2f4-61b3-4252-8c4b-d2113d85d2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:56.278750 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6f2f4-61b3-4252-8c4b-d2113d85d2d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:56.279840 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e6f2f4-61b3-4252-8c4b-d2113d85d2d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:56.281935 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e6f2f4-61b3-4252-8c4b-d2113d85d2d8 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:56.304725 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 895/1782] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:56.318976 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e6f2f4-61b3-4252-8c4b-d2113d85d2d8 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:56.347244 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e6f2f4-61b3-4252-8c4b-d2113d85d2d8 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:56.353407 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5dec3f-ba9b-49f4-b640-9fee72bbdcc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:56.354316 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5dec3f-ba9b-49f4-b640-9fee72bbdcc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:56.354771 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5dec3f-ba9b-49f4-b640-9fee72bbdcc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:56.355280 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5dec3f-ba9b-49f4-b640-9fee72bbdcc9 admin admin] PATH_INFO: `/v3/projects/8882297dcd374e30bccb1ecb34f7b6c0/users/efba360b4cb042b0aec90295ae1d951a/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:56.391216 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5dec3f-ba9b-49f4-b640-9fee72bbdcc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8882297dcd374e30bccb1ecb34f7b6c0, user_id=efba360b4cb042b0aec90295ae1d951a, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:56.404384 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5dec3f-ba9b-49f4-b640-9fee72bbdcc9 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:56.456101 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 888/1783] 10.222.0.55 () {58 vars in 1015 bytes} [Sun Sep 4 01:06:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:56.607884 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 896/1784] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:06:56 2022] PUT /identity/v3/projects/8882297dcd374e30bccb1ecb34f7b6c0/users/efba360b4cb042b0aec90295ae1d951a/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:56.622131 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635cfe6-f2fd-4663-8c5c-f7e41ebf1a30 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:56.623556 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635cfe6-f2fd-4663-8c5c-f7e41ebf1a30 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:56.624246 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635cfe6-f2fd-4663-8c5c-f7e41ebf1a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:56.718018 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-0635cfe6-f2fd-4663-8c5c-f7e41ebf1a30 None None] MFA Rules not processed for user `efba360b4cb042b0aec90295ae1d951a`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:56.724489 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0635cfe6-f2fd-4663-8c5c-f7e41ebf1a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:56.857867 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 889/1785] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:56.863914 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5dec3f-ba9b-49f4-b640-9fee72bbdcc9 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:56.922799 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:56.946773 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635cfe6-f2fd-4663-8c5c-f7e41ebf1a30 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:56.949824 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0635cfe6-f2fd-4663-8c5c-f7e41ebf1a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:57.186558 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5dec3f-ba9b-49f4-b640-9fee72bbdcc9 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:57.233658 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635cfe6-f2fd-4663-8c5c-f7e41ebf1a30 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:57.248650 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be2ada-ee6f-4aac-ab8c-45098d963656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:57.249722 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be2ada-ee6f-4aac-ab8c-45098d963656 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:57.253954 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be2ada-ee6f-4aac-ab8c-45098d963656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:57.255335 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be2ada-ee6f-4aac-ab8c-45098d963656 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:57.256195 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4be2ada-ee6f-4aac-ab8c-45098d963656 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:57.286782 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de1afd-9c87-4dc0-b3e1-b964fe744276 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:57.286782 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de1afd-9c87-4dc0-b3e1-b964fe744276 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:57.286782 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de1afd-9c87-4dc0-b3e1-b964fe744276 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:57.286782 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de1afd-9c87-4dc0-b3e1-b964fe744276 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:57.287303 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4be2ada-ee6f-4aac-ab8c-45098d963656 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:57.316917 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33de1afd-9c87-4dc0-b3e1-b964fe744276 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:57.324141 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33de1afd-9c87-4dc0-b3e1-b964fe744276 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:57.354028 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 890/1786] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:06:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 01:06:57.355019 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de1afd-9c87-4dc0-b3e1-b964fe744276 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:57.360116 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-33de1afd-9c87-4dc0-b3e1-b964fe744276 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:57.408640 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de1afd-9c87-4dc0-b3e1-b964fe744276 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:57.427321 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882552d2-9569-4c0f-b17e-8ff89b6357c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:57.429440 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882552d2-9569-4c0f-b17e-8ff89b6357c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:57.429734 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882552d2-9569-4c0f-b17e-8ff89b6357c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:57.430014 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882552d2-9569-4c0f-b17e-8ff89b6357c8 admin admin] PATH_INFO: `/v3/projects/aac24051b0784e59ac1c0111f4f579d1` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:57.435208 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882552d2-9569-4c0f-b17e-8ff89b6357c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aac24051b0784e59ac1c0111f4f579d1)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:57.437191 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882552d2-9569-4c0f-b17e-8ff89b6357c8 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:57.502708 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 897/1787] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:06:56 2022] POST /identity/v3/projects => generated 381 bytes in 638 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:06:57.505218 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be2ada-ee6f-4aac-ab8c-45098d963656 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:57.738819 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be2ada-ee6f-4aac-ab8c-45098d963656 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:57.766697 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-882552d2-9569-4c0f-b17e-8ff89b6357c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aac24051b0784e59ac1c0111f4f579d1', 'request_id': 'req-882552d2-9569-4c0f-b17e-8ff89b6357c8'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:57.882308 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a04cbbf-4e38-4a79-ab4a-56b73fcee745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:57.883221 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04cbbf-4e38-4a79-ab4a-56b73fcee745 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:57.883717 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04cbbf-4e38-4a79-ab4a-56b73fcee745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:57.884158 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04cbbf-4e38-4a79-ab4a-56b73fcee745 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:57.885032 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a04cbbf-4e38-4a79-ab4a-56b73fcee745 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:57.888639 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-882552d2-9569-4c0f-b17e-8ff89b6357c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aac24051b0784e59ac1c0111f4f579d1', 'request_id': 'req-882552d2-9569-4c0f-b17e-8ff89b6357c8'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:57.914214 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a04cbbf-4e38-4a79-ab4a-56b73fcee745 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:57.914971 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-882552d2-9569-4c0f-b17e-8ff89b6357c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aac24051b0784e59ac1c0111f4f579d1', 'request_id': 'req-882552d2-9569-4c0f-b17e-8ff89b6357c8'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:57.914971 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-882552d2-9569-4c0f-b17e-8ff89b6357c8 admin admin] The token cache is being invalidate because project aac24051b0784e59ac1c0111f4f579d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:06:57.914971 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-882552d2-9569-4c0f-b17e-8ff89b6357c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:57.924838 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-882552d2-9569-4c0f-b17e-8ff89b6357c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:57.930375 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 891/1788] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:06:57 2022] DELETE /identity/v3/projects/aac24051b0784e59ac1c0111f4f579d1 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:57.932874 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882552d2-9569-4c0f-b17e-8ff89b6357c8 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:57.935411 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-882552d2-9569-4c0f-b17e-8ff89b6357c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:57.976365 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882552d2-9569-4c0f-b17e-8ff89b6357c8 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:57.997401 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b86f83-a081-454f-bb5f-a5c3216f0744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:57.997625 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b86f83-a081-454f-bb5f-a5c3216f0744 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:57.998106 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b86f83-a081-454f-bb5f-a5c3216f0744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:57.998211 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b86f83-a081-454f-bb5f-a5c3216f0744 admin admin] PATH_INFO: `/v3/users/1f8b1011c2b24f3b85e8cfb28aea8f49` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:58.002042 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b86f83-a081-454f-bb5f-a5c3216f0744 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f8b1011c2b24f3b85e8cfb28aea8f49)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:58.004062 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b86f83-a081-454f-bb5f-a5c3216f0744 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:58.150259 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 898/1789] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:06:57 2022] POST /identity/v3/users => generated 297 bytes in 647 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:58.152964 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a04cbbf-4e38-4a79-ab4a-56b73fcee745 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:58.156070 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-6a04cbbf-4e38-4a79-ab4a-56b73fcee745 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:58.203726 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a04cbbf-4e38-4a79-ab4a-56b73fcee745 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:58.225916 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f77f872-0545-4e39-adc1-949d8a48d800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:58.226727 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f77f872-0545-4e39-adc1-949d8a48d800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:58.227014 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f77f872-0545-4e39-adc1-949d8a48d800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:58.227289 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f77f872-0545-4e39-adc1-949d8a48d800 admin admin] PATH_INFO: `/v3/projects/2a59f67920fb433c8018ec4d98a87b8a` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:58.233543 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f77f872-0545-4e39-adc1-949d8a48d800 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a59f67920fb433c8018ec4d98a87b8a)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:58.240060 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f77f872-0545-4e39-adc1-949d8a48d800 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:58.318789 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:58.359601 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d5b86f83-a081-454f-bb5f-a5c3216f0744 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f8b1011c2b24f3b85e8cfb28aea8f49'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:58.366990 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d5b86f83-a081-454f-bb5f-a5c3216f0744 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f8b1011c2b24f3b85e8cfb28aea8f49'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:58.416776 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d5b86f83-a081-454f-bb5f-a5c3216f0744 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f8b1011c2b24f3b85e8cfb28aea8f49'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:58.609818 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d5b86f83-a081-454f-bb5f-a5c3216f0744 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f8b1011c2b24f3b85e8cfb28aea8f49'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:58.610793 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d5b86f83-a081-454f-bb5f-a5c3216f0744 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f8b1011c2b24f3b85e8cfb28aea8f49'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:58.619443 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1f77f872-0545-4e39-adc1-949d8a48d800 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a59f67920fb433c8018ec4d98a87b8a', 'request_id': 'req-1f77f872-0545-4e39-adc1-949d8a48d800'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:58.654187 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 892/1790] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:06:57 2022] DELETE /identity/v3/users/1f8b1011c2b24f3b85e8cfb28aea8f49 => generated 0 bytes in 723 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:58.655482 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b86f83-a081-454f-bb5f-a5c3216f0744 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:58.656994 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d5b86f83-a081-454f-bb5f-a5c3216f0744 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:58.774173 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b86f83-a081-454f-bb5f-a5c3216f0744 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:58.793131 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8394fbf6-d5a9-4abe-8bfa-2b450cd8a7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:58.794521 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394fbf6-d5a9-4abe-8bfa-2b450cd8a7c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:58.795351 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394fbf6-d5a9-4abe-8bfa-2b450cd8a7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:58.796114 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394fbf6-d5a9-4abe-8bfa-2b450cd8a7c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:58.797693 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8394fbf6-d5a9-4abe-8bfa-2b450cd8a7c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:58.801761 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8394fbf6-d5a9-4abe-8bfa-2b450cd8a7c1 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:58.830630 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 893/1791] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:58.830630 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8394fbf6-d5a9-4abe-8bfa-2b450cd8a7c1 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:58.849105 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1f77f872-0545-4e39-adc1-949d8a48d800 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a59f67920fb433c8018ec4d98a87b8a', 'request_id': 'req-1f77f872-0545-4e39-adc1-949d8a48d800'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:58.854900 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8394fbf6-d5a9-4abe-8bfa-2b450cd8a7c1 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:58.861880 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1f77f872-0545-4e39-adc1-949d8a48d800 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a59f67920fb433c8018ec4d98a87b8a', 'request_id': 'req-1f77f872-0545-4e39-adc1-949d8a48d800'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:58.900884 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f146f-ccf4-4052-8a5d-def6c217056e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:58.901806 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1f77f872-0545-4e39-adc1-949d8a48d800 admin admin] The token cache is being invalidate because project 2a59f67920fb433c8018ec4d98a87b8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:06:58.902206 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1f77f872-0545-4e39-adc1-949d8a48d800 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:58.903370 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-1f77f872-0545-4e39-adc1-949d8a48d800 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:06:58.906057 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f146f-ccf4-4052-8a5d-def6c217056e admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:58.906402 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f146f-ccf4-4052-8a5d-def6c217056e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:58.906960 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 899/1792] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:06:58 2022] DELETE /identity/v3/projects/2a59f67920fb433c8018ec4d98a87b8a => generated 0 bytes in 756 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:58.907469 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f146f-ccf4-4052-8a5d-def6c217056e admin admin] PATH_INFO: `/v3/projects/a839cb9e24574e21bff08a1a05fd9f93/users/39c101b9c9a14c7db866db9906026f7e/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:58.912996 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970883f9-885c-4f9a-91a7-39e0afc486a1 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:58.913693 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970883f9-885c-4f9a-91a7-39e0afc486a1 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:58.914157 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970883f9-885c-4f9a-91a7-39e0afc486a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:58.983886 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f146f-ccf4-4052-8a5d-def6c217056e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a839cb9e24574e21bff08a1a05fd9f93, user_id=39c101b9c9a14c7db866db9906026f7e, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:58.984959 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-970883f9-885c-4f9a-91a7-39e0afc486a1 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:58.994040 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-970883f9-885c-4f9a-91a7-39e0afc486a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:58.994313 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f146f-ccf4-4052-8a5d-def6c217056e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:59.016772 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 900/1793] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:06:59.098139 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8875fa5-41b6-4740-bb14-b306022d188e None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:59.099178 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8875fa5-41b6-4740-bb14-b306022d188e None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:59.100200 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8875fa5-41b6-4740-bb14-b306022d188e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:59.165195 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-d8875fa5-41b6-4740-bb14-b306022d188e None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:59.165195 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d8875fa5-41b6-4740-bb14-b306022d188e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:59.194238 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 901/1794] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:06:59.197410 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8875fa5-41b6-4740-bb14-b306022d188e None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:59.242944 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8875fa5-41b6-4740-bb14-b306022d188e None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:59.260163 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7045d9-f178-40d8-9979-c289d52cf58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:59.260866 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7045d9-f178-40d8-9979-c289d52cf58c admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:59.261134 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7045d9-f178-40d8-9979-c289d52cf58c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:59.261414 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7045d9-f178-40d8-9979-c289d52cf58c admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:59.262099 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7045d9-f178-40d8-9979-c289d52cf58c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:06:59.262408 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7045d9-f178-40d8-9979-c289d52cf58c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:59.264369 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 894/1795] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:06:58 2022] PUT /identity/v3/projects/a839cb9e24574e21bff08a1a05fd9f93/users/39c101b9c9a14c7db866db9906026f7e/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:59.266202 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f146f-ccf4-4052-8a5d-def6c217056e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:59.266330 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7045d9-f178-40d8-9979-c289d52cf58c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:59.269808 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-468f146f-ccf4-4052-8a5d-def6c217056e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:59.301228 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 902/1796] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:06:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:06:59.302174 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7045d9-f178-40d8-9979-c289d52cf58c admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:59.331648 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7045d9-f178-40d8-9979-c289d52cf58c admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:59.473414 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f146f-ccf4-4052-8a5d-def6c217056e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:59.478797 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46904667-b3a7-4b8c-ac55-c4a6d3325120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:59.480288 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46904667-b3a7-4b8c-ac55-c4a6d3325120 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:59.480564 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46904667-b3a7-4b8c-ac55-c4a6d3325120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:59.480841 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46904667-b3a7-4b8c-ac55-c4a6d3325120 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:59.481493 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46904667-b3a7-4b8c-ac55-c4a6d3325120 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:59.483497 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1587a1-02ae-45a2-a650-ef73969025fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:59.484319 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46904667-b3a7-4b8c-ac55-c4a6d3325120 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:59.485142 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1587a1-02ae-45a2-a650-ef73969025fe admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:59.488016 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1587a1-02ae-45a2-a650-ef73969025fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:59.488505 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1587a1-02ae-45a2-a650-ef73969025fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:59.489354 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1587a1-02ae-45a2-a650-ef73969025fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:59.494295 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1587a1-02ae-45a2-a650-ef73969025fe admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:59.558124 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 895/1797] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:06:59.559953 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1587a1-02ae-45a2-a650-ef73969025fe admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:59.585573 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1587a1-02ae-45a2-a650-ef73969025fe admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:59.616636 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a32467-190d-42f7-adaf-b30aed22cf21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:59.625502 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a32467-190d-42f7-adaf-b30aed22cf21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:59.625502 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a32467-190d-42f7-adaf-b30aed22cf21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:59.625502 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a32467-190d-42f7-adaf-b30aed22cf21 admin admin] PATH_INFO: `/v3/projects/a839cb9e24574e21bff08a1a05fd9f93/users/39c101b9c9a14c7db866db9906026f7e/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:59.635082 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a32467-190d-42f7-adaf-b30aed22cf21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a839cb9e24574e21bff08a1a05fd9f93, user_id=39c101b9c9a14c7db866db9906026f7e, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:59.641328 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 903/1798] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:06:59 2022] POST /identity/v3/projects => generated 373 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:06:59.645957 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46904667-b3a7-4b8c-ac55-c4a6d3325120 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:59.647913 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a32467-190d-42f7-adaf-b30aed22cf21 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:06:59.695431 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 896/1799] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:06:59 2022] PUT /identity/v3/projects/a839cb9e24574e21bff08a1a05fd9f93/users/39c101b9c9a14c7db866db9906026f7e/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:06:59.702020 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:59.709023 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9216a-2514-4fc1-91ee-dab96d5f62cc None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:59.763453 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9216a-2514-4fc1-91ee-dab96d5f62cc None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:59.763453 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9216a-2514-4fc1-91ee-dab96d5f62cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:59.763453 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1ec9216a-2514-4fc1-91ee-dab96d5f62cc None None] MFA Rules not processed for user `39c101b9c9a14c7db866db9906026f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:06:59.763453 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1ec9216a-2514-4fc1-91ee-dab96d5f62cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:59.812405 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46904667-b3a7-4b8c-ac55-c4a6d3325120 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:06:59.911251 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 897/1800] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:06:59 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:06:59.923111 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9216a-2514-4fc1-91ee-dab96d5f62cc None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:06:59.929304 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1ec9216a-2514-4fc1-91ee-dab96d5f62cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:06:59.978193 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac44295-59f2-44e7-887a-3076f3e9d20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:06:59.980307 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac44295-59f2-44e7-887a-3076f3e9d20b admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:06:59.980307 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac44295-59f2-44e7-887a-3076f3e9d20b admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:06:59.982479 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac44295-59f2-44e7-887a-3076f3e9d20b admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:06:59.984351 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac44295-59f2-44e7-887a-3076f3e9d20b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:06:59.993141 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac44295-59f2-44e7-887a-3076f3e9d20b admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:00.056851 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9216a-2514-4fc1-91ee-dab96d5f62cc None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:00.062532 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f2011-09f6-44ab-8704-3d49a9c37960 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:00.064059 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f2011-09f6-44ab-8704-3d49a9c37960 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:00.065082 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f2011-09f6-44ab-8704-3d49a9c37960 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:00.065453 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f2011-09f6-44ab-8704-3d49a9c37960 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:00.082896 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0f2011-09f6-44ab-8704-3d49a9c37960 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:00.090225 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0f2011-09f6-44ab-8704-3d49a9c37960 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:00.100176 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 898/1801] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:06:59 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:07:00.118794 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 904/1802] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:06:59 2022] POST /identity/v3/users => generated 287 bytes in 472 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:07:00.123172 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f2011-09f6-44ab-8704-3d49a9c37960 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:00.153671 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f2011-09f6-44ab-8704-3d49a9c37960 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:00.178360 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c26e2-f927-4b29-8ad9-d9b0402597ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:00.179070 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c26e2-f927-4b29-8ad9-d9b0402597ab admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:00.179885 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c26e2-f927-4b29-8ad9-d9b0402597ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:00.179885 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c26e2-f927-4b29-8ad9-d9b0402597ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:00.180512 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562c26e2-f927-4b29-8ad9-d9b0402597ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:00.192646 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562c26e2-f927-4b29-8ad9-d9b0402597ab admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:00.218362 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 899/1803] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:00.223181 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac44295-59f2-44e7-887a-3076f3e9d20b admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:00.241312 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac44295-59f2-44e7-887a-3076f3e9d20b admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:00.259193 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c02be-c59c-49b7-ad95-c2a9f1a0cf1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:00.284631 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c02be-c59c-49b7-ad95-c2a9f1a0cf1a admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:00.284631 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c02be-c59c-49b7-ad95-c2a9f1a0cf1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:00.284631 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c02be-c59c-49b7-ad95-c2a9f1a0cf1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:00.284631 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c02be-c59c-49b7-ad95-c2a9f1a0cf1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:00.284631 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c02be-c59c-49b7-ad95-c2a9f1a0cf1a admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:00.286148 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 905/1804] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:00.287444 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c26e2-f927-4b29-8ad9-d9b0402597ab admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:00.301932 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c02be-c59c-49b7-ad95-c2a9f1a0cf1a admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:00.304808 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c26e2-f927-4b29-8ad9-d9b0402597ab admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:00.333879 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c02be-c59c-49b7-ad95-c2a9f1a0cf1a admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:00.337238 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c069f6-118c-4826-8ced-f19e48237dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:00.338105 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c069f6-118c-4826-8ced-f19e48237dba admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:00.338540 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c069f6-118c-4826-8ced-f19e48237dba admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:00.338954 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c069f6-118c-4826-8ced-f19e48237dba admin admin] PATH_INFO: `/v3/projects/e8b6424a4ded4345bfca3379d4a3d718/users/b8551a2cd0d042ec83cd1afc8cc721b3/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:00.339835 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b098f-f685-482b-bc59-a5b03b569768 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:00.341746 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b098f-f685-482b-bc59-a5b03b569768 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:00.342122 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b098f-f685-482b-bc59-a5b03b569768 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:00.342480 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b098f-f685-482b-bc59-a5b03b569768 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:00.344700 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b75b098f-f685-482b-bc59-a5b03b569768 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:00.375270 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c069f6-118c-4826-8ced-f19e48237dba admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8b6424a4ded4345bfca3379d4a3d718, user_id=b8551a2cd0d042ec83cd1afc8cc721b3, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:00.378936 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c069f6-118c-4826-8ced-f19e48237dba admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:00.475846 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b098f-f685-482b-bc59-a5b03b569768 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:00.479134 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b098f-f685-482b-bc59-a5b03b569768 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:00.487719 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 900/1805] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:00 2022] PUT /identity/v3/projects/e8b6424a4ded4345bfca3379d4a3d718/users/b8551a2cd0d042ec83cd1afc8cc721b3/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:00.499253 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75271735-79ec-47d6-9c22-d07db0eab1a7 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:00.499772 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75271735-79ec-47d6-9c22-d07db0eab1a7 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:00.500213 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75271735-79ec-47d6-9c22-d07db0eab1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:00.501032 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 906/1806] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:07:00.606979 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-75271735-79ec-47d6-9c22-d07db0eab1a7 None None] MFA Rules not processed for user `b8551a2cd0d042ec83cd1afc8cc721b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:00.608228 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-75271735-79ec-47d6-9c22-d07db0eab1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:00.786224 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 901/1807] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:00.789986 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b098f-f685-482b-bc59-a5b03b569768 service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:00.823713 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b098f-f685-482b-bc59-a5b03b569768 service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:01.038036 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75271735-79ec-47d6-9c22-d07db0eab1a7 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:01.040119 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-75271735-79ec-47d6-9c22-d07db0eab1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:01.063541 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48949277-6395-434a-bc87-9248de000e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:01.064224 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48949277-6395-434a-bc87-9248de000e8f admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:01.072592 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48949277-6395-434a-bc87-9248de000e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:01.073072 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48949277-6395-434a-bc87-9248de000e8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:01.073832 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48949277-6395-434a-bc87-9248de000e8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:01.084186 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48949277-6395-434a-bc87-9248de000e8f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:01.109247 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75271735-79ec-47d6-9c22-d07db0eab1a7 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:01.139649 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e247867c-cf05-40e6-9090-92408f47107c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:01.140330 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e247867c-cf05-40e6-9090-92408f47107c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:01.140671 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e247867c-cf05-40e6-9090-92408f47107c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:01.140954 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e247867c-cf05-40e6-9090-92408f47107c admin admin] PATH_INFO: `/v3/projects/66780a4062684b0199c9c3ebceb9af8d` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:01.147782 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e247867c-cf05-40e6-9090-92408f47107c admin admin] RBAC: Authorizing `identity:delete_project(project_id=66780a4062684b0199c9c3ebceb9af8d)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:01.156190 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:01.160660 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e247867c-cf05-40e6-9090-92408f47107c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:01.243978 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 907/1808] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:07:00 2022] POST /identity/v3/projects => generated 371 bytes in 455 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:01.250413 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48949277-6395-434a-bc87-9248de000e8f admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:01.313682 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:01.385938 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e247867c-cf05-40e6-9090-92408f47107c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66780a4062684b0199c9c3ebceb9af8d', 'request_id': 'req-e247867c-cf05-40e6-9090-92408f47107c'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:01.409869 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48949277-6395-434a-bc87-9248de000e8f admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:01.482779 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e247867c-cf05-40e6-9090-92408f47107c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66780a4062684b0199c9c3ebceb9af8d', 'request_id': 'req-e247867c-cf05-40e6-9090-92408f47107c'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:01.484420 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e247867c-cf05-40e6-9090-92408f47107c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66780a4062684b0199c9c3ebceb9af8d', 'request_id': 'req-e247867c-cf05-40e6-9090-92408f47107c'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:01.491481 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd78042-004f-44d7-9179-d21a85eb705e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:01.494183 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd78042-004f-44d7-9179-d21a85eb705e admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:01.494475 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd78042-004f-44d7-9179-d21a85eb705e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:01.494765 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd78042-004f-44d7-9179-d21a85eb705e admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:01.495420 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd78042-004f-44d7-9179-d21a85eb705e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:01.502366 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd78042-004f-44d7-9179-d21a85eb705e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:01.518624 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e247867c-cf05-40e6-9090-92408f47107c admin admin] The token cache is being invalidate because project 66780a4062684b0199c9c3ebceb9af8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:07:01.518967 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e247867c-cf05-40e6-9090-92408f47107c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:01.522788 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e247867c-cf05-40e6-9090-92408f47107c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:01.526054 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 902/1809] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:07:01 2022] DELETE /identity/v3/projects/66780a4062684b0199c9c3ebceb9af8d => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:07:01.546474 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbae93-568c-41de-bfb2-e8b6cdcd56a7 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:01.547272 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbae93-568c-41de-bfb2-e8b6cdcd56a7 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:01.547272 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbae93-568c-41de-bfb2-e8b6cdcd56a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:01.577203 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-ffcbae93-568c-41de-bfb2-e8b6cdcd56a7 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:01.578955 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ffcbae93-568c-41de-bfb2-e8b6cdcd56a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:01.597044 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 903/1810] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:01.598877 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b467e-e775-4c18-8028-abad8730dea0 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:01.599961 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b467e-e775-4c18-8028-abad8730dea0 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:01.600383 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b467e-e775-4c18-8028-abad8730dea0 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:01.607393 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 904/1811] 10.222.0.27 () {54 vars in 921 bytes} [Sun Sep 4 01:07:01 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:07:01.613513 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad02f00-781d-473b-98cb-91a0db75e15b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:01.617713 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad02f00-781d-473b-98cb-91a0db75e15b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:01.618160 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad02f00-781d-473b-98cb-91a0db75e15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:01.660394 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-9ad02f00-781d-473b-98cb-91a0db75e15b None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:01.661919 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9ad02f00-781d-473b-98cb-91a0db75e15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:01.671631 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 905/1812] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:01.677197 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad02f00-781d-473b-98cb-91a0db75e15b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:01.690929 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad02f00-781d-473b-98cb-91a0db75e15b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:01.701309 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f49d8a-9462-4137-82e5-bd991c7a7610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:01.708385 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f49d8a-9462-4137-82e5-bd991c7a7610 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:01.708385 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f49d8a-9462-4137-82e5-bd991c7a7610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:01.708385 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f49d8a-9462-4137-82e5-bd991c7a7610 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:01.708385 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f49d8a-9462-4137-82e5-bd991c7a7610 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:07:01.708385 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f49d8a-9462-4137-82e5-bd991c7a7610 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:01.708385 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f49d8a-9462-4137-82e5-bd991c7a7610 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:01.730300 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 906/1813] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:07:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:07:01.732719 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50a314-9913-4ad9-b6c3-e4cbdc4d73aa None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:01.736879 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50a314-9913-4ad9-b6c3-e4cbdc4d73aa None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:01.736879 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50a314-9913-4ad9-b6c3-e4cbdc4d73aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:01.764278 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 908/1814] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:07:01 2022] POST /identity/v3/users => generated 292 bytes in 512 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:01.764278 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd78042-004f-44d7-9179-d21a85eb705e admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:01.785059 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-0d50a314-9913-4ad9-b6c3-e4cbdc4d73aa None None] MFA Rules not processed for user `68bc00bda5194cd8a7928f8c4df4ccd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:01.787793 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0d50a314-9913-4ad9-b6c3-e4cbdc4d73aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:01.788996 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd78042-004f-44d7-9179-d21a85eb705e admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:01.802808 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e1706-692b-43a0-845d-dc0de3193fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:01.804816 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e1706-692b-43a0-845d-dc0de3193fac admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:01.810780 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e1706-692b-43a0-845d-dc0de3193fac admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:01.810780 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e1706-692b-43a0-845d-dc0de3193fac admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:01.810780 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e1706-692b-43a0-845d-dc0de3193fac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:01.810780 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e1706-692b-43a0-845d-dc0de3193fac admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:01.938984 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 909/1815] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:07:01 2022] POST /identity/v3/projects => generated 391 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:01.940419 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e1706-692b-43a0-845d-dc0de3193fac admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:01.962053 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f64e1706-692b-43a0-845d-dc0de3193fac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:01.994501 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:02.013191 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 907/1816] 10.222.0.27 () {58 vars in 1018 bytes} [Sun Sep 4 01:07:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:02.014283 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50a314-9913-4ad9-b6c3-e4cbdc4d73aa None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:02.027239 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50a314-9913-4ad9-b6c3-e4cbdc4d73aa None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:02.072264 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e1706-692b-43a0-845d-dc0de3193fac admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:02.078673 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00155c-0807-4fb1-8e7b-bb76091638ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:02.079528 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00155c-0807-4fb1-8e7b-bb76091638ac admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:02.080579 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00155c-0807-4fb1-8e7b-bb76091638ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:02.081987 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00155c-0807-4fb1-8e7b-bb76091638ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:02.082932 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a00155c-0807-4fb1-8e7b-bb76091638ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:02.085634 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a00155c-0807-4fb1-8e7b-bb76091638ac admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:02.100720 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 910/1817] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:02.102374 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc966e48-2018-4bd4-af6d-9ea99b87faaf None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:02.102828 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc966e48-2018-4bd4-af6d-9ea99b87faaf None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:02.103264 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc966e48-2018-4bd4-af6d-9ea99b87faaf None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:02.104989 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 911/1818] 10.222.0.27 () {54 vars in 921 bytes} [Sun Sep 4 01:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:07:02.105891 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc966e48-2018-4bd4-af6d-9ea99b87faaf None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:02.121255 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1a65f6-fa1b-423d-a3e8-538588985c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:02.122641 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a65f6-fa1b-423d-a3e8-538588985c85 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:02.123112 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a65f6-fa1b-423d-a3e8-538588985c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:02.123567 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a65f6-fa1b-423d-a3e8-538588985c85 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:02.124388 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1a65f6-fa1b-423d-a3e8-538588985c85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:02.132129 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1a65f6-fa1b-423d-a3e8-538588985c85 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:02.157340 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc966e48-2018-4bd4-af6d-9ea99b87faaf None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:02.181131 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac0572-fb0f-4791-83c8-88c92bd6e8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:02.183384 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac0572-fb0f-4791-83c8-88c92bd6e8ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:02.183820 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac0572-fb0f-4791-83c8-88c92bd6e8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:02.184916 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac0572-fb0f-4791-83c8-88c92bd6e8ba admin admin] PATH_INFO: `/v3/projects/fd38e4cafd2343b696701e1815b10fdd/users/6d9a7cf18e7d40678501b1ea47c060af/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:02.259294 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac0572-fb0f-4791-83c8-88c92bd6e8ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd38e4cafd2343b696701e1815b10fdd, user_id=6d9a7cf18e7d40678501b1ea47c060af, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:02.265642 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac0572-fb0f-4791-83c8-88c92bd6e8ba admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:02.372776 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 908/1819] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:07:02 2022] POST /identity/v3/users => generated 296 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:02.374473 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84e413-ba4a-4614-a00c-48d2b09c389d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:02.374920 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84e413-ba4a-4614-a00c-48d2b09c389d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:02.375205 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84e413-ba4a-4614-a00c-48d2b09c389d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:02.407570 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 912/1820] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:02 2022] PUT /identity/v3/projects/fd38e4cafd2343b696701e1815b10fdd/users/6d9a7cf18e7d40678501b1ea47c060af/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:02.411852 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac0572-fb0f-4791-83c8-88c92bd6e8ba admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:02.417020 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-16ac0572-fb0f-4791-83c8-88c92bd6e8ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:02.475632 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-bf84e413-ba4a-4614-a00c-48d2b09c389d None None] MFA Rules not processed for user `68bc00bda5194cd8a7928f8c4df4ccd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:02.475632 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bf84e413-ba4a-4614-a00c-48d2b09c389d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:02.649255 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 909/1821] 10.222.0.27 () {58 vars in 1018 bytes} [Sun Sep 4 01:07:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:02.650508 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84e413-ba4a-4614-a00c-48d2b09c389d None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:02.739252 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac0572-fb0f-4791-83c8-88c92bd6e8ba admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:02.761129 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1feec5a-45fd-4322-9fa5-b1309e3eb96d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:02.763726 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1feec5a-45fd-4322-9fa5-b1309e3eb96d service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:02.770445 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1feec5a-45fd-4322-9fa5-b1309e3eb96d service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:02.770445 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1feec5a-45fd-4322-9fa5-b1309e3eb96d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:02.809670 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1feec5a-45fd-4322-9fa5-b1309e3eb96d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:02.816868 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1feec5a-45fd-4322-9fa5-b1309e3eb96d service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:02.828157 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84e413-ba4a-4614-a00c-48d2b09c389d None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:02.838507 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af06b5b-46d0-41f0-9b3e-fd4b3bd82d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:02.839827 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af06b5b-46d0-41f0-9b3e-fd4b3bd82d37 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:02.839827 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af06b5b-46d0-41f0-9b3e-fd4b3bd82d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:02.839827 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af06b5b-46d0-41f0-9b3e-fd4b3bd82d37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:02.840522 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af06b5b-46d0-41f0-9b3e-fd4b3bd82d37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:02.842194 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af06b5b-46d0-41f0-9b3e-fd4b3bd82d37 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:02.844044 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 913/1822] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:07:02.848787 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1feec5a-45fd-4322-9fa5-b1309e3eb96d service nova] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:02.865970 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1feec5a-45fd-4322-9fa5-b1309e3eb96d service nova] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:02.883842 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7beefff-cc12-47c7-8b13-5156c2987a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:02.885125 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7beefff-cc12-47c7-8b13-5156c2987a09 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:02.887084 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7beefff-cc12-47c7-8b13-5156c2987a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:02.887465 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7beefff-cc12-47c7-8b13-5156c2987a09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:02.888581 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7beefff-cc12-47c7-8b13-5156c2987a09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:02.889599 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 910/1823] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:02.891716 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7beefff-cc12-47c7-8b13-5156c2987a09 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:02.894847 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af06b5b-46d0-41f0-9b3e-fd4b3bd82d37 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:02.914135 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af06b5b-46d0-41f0-9b3e-fd4b3bd82d37 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:02.919190 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 914/1824] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:02.922774 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7beefff-cc12-47c7-8b13-5156c2987a09 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:02.943961 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7beefff-cc12-47c7-8b13-5156c2987a09 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:02.958287 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688d3df-8965-460f-bf36-76c1012ab21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:02.958287 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688d3df-8965-460f-bf36-76c1012ab21d admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:02.958287 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688d3df-8965-460f-bf36-76c1012ab21d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:02.958287 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688d3df-8965-460f-bf36-76c1012ab21d admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:02.958965 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688d3df-8965-460f-bf36-76c1012ab21d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:02.958965 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688d3df-8965-460f-bf36-76c1012ab21d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:02.959134 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c3a21-9d1d-46a6-b14c-3d01d5742375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:02.959374 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c3a21-9d1d-46a6-b14c-3d01d5742375 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:02.959659 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c3a21-9d1d-46a6-b14c-3d01d5742375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:02.959931 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c3a21-9d1d-46a6-b14c-3d01d5742375 admin admin] PATH_INFO: `/v3/projects/fd38e4cafd2343b696701e1815b10fdd/users/6d9a7cf18e7d40678501b1ea47c060af/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:02.969393 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91c3a21-9d1d-46a6-b14c-3d01d5742375 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd38e4cafd2343b696701e1815b10fdd, user_id=6d9a7cf18e7d40678501b1ea47c060af, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:02.972259 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 911/1825] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 01:07:02.975670 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91c3a21-9d1d-46a6-b14c-3d01d5742375 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:02.976175 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688d3df-8965-460f-bf36-76c1012ab21d admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:02.996252 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688d3df-8965-460f-bf36-76c1012ab21d admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:03.012863 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 915/1826] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:02 2022] PUT /identity/v3/projects/fd38e4cafd2343b696701e1815b10fdd/users/6d9a7cf18e7d40678501b1ea47c060af/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:03.029374 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16207b48-ab07-41d2-ac77-08ed5acfe4e7 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:03.029686 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16207b48-ab07-41d2-ac77-08ed5acfe4e7 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:03.029934 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16207b48-ab07-41d2-ac77-08ed5acfe4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:03.124557 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-16207b48-ab07-41d2-ac77-08ed5acfe4e7 None None] MFA Rules not processed for user `6d9a7cf18e7d40678501b1ea47c060af`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:03.124557 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-16207b48-ab07-41d2-ac77-08ed5acfe4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:03.191702 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f9b36f-1bd8-4e7f-b65e-d236b6432514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:03.205877 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9b36f-1bd8-4e7f-b65e-d236b6432514 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:03.205877 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9b36f-1bd8-4e7f-b65e-d236b6432514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:03.205877 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9b36f-1bd8-4e7f-b65e-d236b6432514 admin admin] PATH_INFO: `/v3/projects/66fb024fe3eb4bca99c6ddb79d4e9f37/users/3e59e061bb514bd8b3746da7b0a88058/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:03.218354 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f9b36f-1bd8-4e7f-b65e-d236b6432514 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66fb024fe3eb4bca99c6ddb79d4e9f37, user_id=3e59e061bb514bd8b3746da7b0a88058, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:03.223188 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f9b36f-1bd8-4e7f-b65e-d236b6432514 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:03.272078 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 916/1827] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:03 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:03.273455 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16207b48-ab07-41d2-ac77-08ed5acfe4e7 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:03.357541 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 912/1828] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:02 2022] PUT /identity/v3/projects/66fb024fe3eb4bca99c6ddb79d4e9f37/users/3e59e061bb514bd8b3746da7b0a88058/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:07:03.358967 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f9b36f-1bd8-4e7f-b65e-d236b6432514 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:03.363978 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b0f9b36f-1bd8-4e7f-b65e-d236b6432514 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:03.389625 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16207b48-ab07-41d2-ac77-08ed5acfe4e7 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:03.540249 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f9b36f-1bd8-4e7f-b65e-d236b6432514 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:03.541943 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609c775-a4dc-4443-aa2e-aafc0f150604 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:03.543700 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609c775-a4dc-4443-aa2e-aafc0f150604 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:03.544181 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609c775-a4dc-4443-aa2e-aafc0f150604 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:03.545331 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609c775-a4dc-4443-aa2e-aafc0f150604 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:03.553752 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610c92f9-8524-4b03-a1ea-e2032031903f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:03.554871 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610c92f9-8524-4b03-a1ea-e2032031903f service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:03.557508 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610c92f9-8524-4b03-a1ea-e2032031903f service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:03.558217 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610c92f9-8524-4b03-a1ea-e2032031903f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:03.575131 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610c92f9-8524-4b03-a1ea-e2032031903f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:03.578099 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610c92f9-8524-4b03-a1ea-e2032031903f service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:03.657286 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 917/1829] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:07:03.667482 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a7f9b-ad9d-4ab0-8aff-bf15ac56623e None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:03.669243 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a7f9b-ad9d-4ab0-8aff-bf15ac56623e None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:03.669596 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a7f9b-ad9d-4ab0-8aff-bf15ac56623e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:03.832432 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-501a7f9b-ad9d-4ab0-8aff-bf15ac56623e None None] MFA Rules not processed for user `3e59e061bb514bd8b3746da7b0a88058`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:03.839721 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-501a7f9b-ad9d-4ab0-8aff-bf15ac56623e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:03.873326 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2609c775-a4dc-4443-aa2e-aafc0f150604 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:03.874797 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2609c775-a4dc-4443-aa2e-aafc0f150604 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:03.921309 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 913/1830] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:03 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:07:03.976352 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 918/1831] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:03 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:03.981312 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609c775-a4dc-4443-aa2e-aafc0f150604 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:04.133458 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609c775-a4dc-4443-aa2e-aafc0f150604 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:04.139256 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc40bb8a-a285-45af-83dd-cc472d84dbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:04.139471 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40bb8a-a285-45af-83dd-cc472d84dbab admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:04.139854 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40bb8a-a285-45af-83dd-cc472d84dbab admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:04.140197 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40bb8a-a285-45af-83dd-cc472d84dbab admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:04.141660 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc40bb8a-a285-45af-83dd-cc472d84dbab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:04.143403 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc40bb8a-a285-45af-83dd-cc472d84dbab admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:04.285051 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 914/1832] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:07:03 2022] POST /identity/v3/projects => generated 393 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:04.287266 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501a7f9b-ad9d-4ab0-8aff-bf15ac56623e None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:04.306338 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501a7f9b-ad9d-4ab0-8aff-bf15ac56623e None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:04.457443 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce221a5-6f0f-4c40-aeb0-d7d17d74e395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:04.458198 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce221a5-6f0f-4c40-aeb0-d7d17d74e395 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:04.458502 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce221a5-6f0f-4c40-aeb0-d7d17d74e395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:04.458765 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce221a5-6f0f-4c40-aeb0-d7d17d74e395 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:04.459587 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce221a5-6f0f-4c40-aeb0-d7d17d74e395 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:04.465865 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce221a5-6f0f-4c40-aeb0-d7d17d74e395 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:04.612403 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 919/1833] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:07:04 2022] POST /identity/v3/users => generated 303 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:04.613663 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc40bb8a-a285-45af-83dd-cc472d84dbab admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:04.636854 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc40bb8a-a285-45af-83dd-cc472d84dbab admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:04.657132 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a550d-aeac-452d-92e5-7151d3c7aed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:04.658298 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a550d-aeac-452d-92e5-7151d3c7aed4 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:04.659677 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a550d-aeac-452d-92e5-7151d3c7aed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:04.660142 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a550d-aeac-452d-92e5-7151d3c7aed4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:04.661262 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176a550d-aeac-452d-92e5-7151d3c7aed4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:04.667052 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176a550d-aeac-452d-92e5-7151d3c7aed4 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:04.730192 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 915/1834] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:04.731722 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce221a5-6f0f-4c40-aeb0-d7d17d74e395 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:04.749943 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce221a5-6f0f-4c40-aeb0-d7d17d74e395 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:04.757993 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f930940c-27f8-4502-acab-c0d2a3d68270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:04.758903 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930940c-27f8-4502-acab-c0d2a3d68270 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:04.758903 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930940c-27f8-4502-acab-c0d2a3d68270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:04.759099 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930940c-27f8-4502-acab-c0d2a3d68270 admin admin] PATH_INFO: `/v3/projects/e76e4004f076461d8a16cd7b380fb1dc/users/b2e2c37a891245e0ad13258afe7fda1d/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:04.788421 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f930940c-27f8-4502-acab-c0d2a3d68270 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e76e4004f076461d8a16cd7b380fb1dc, user_id=b2e2c37a891245e0ad13258afe7fda1d, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:04.791714 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f930940c-27f8-4502-acab-c0d2a3d68270 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:04.950751 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:05.010943 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 920/1835] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:04 2022] PUT /identity/v3/projects/e76e4004f076461d8a16cd7b380fb1dc/users/b2e2c37a891245e0ad13258afe7fda1d/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:05.012139 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a550d-aeac-452d-92e5-7151d3c7aed4 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:05.024971 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a550d-aeac-452d-92e5-7151d3c7aed4 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:05.137865 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf04ab1-ce0c-4699-ab82-8de7892f624f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:05.138510 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf04ab1-ce0c-4699-ab82-8de7892f624f admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:05.139052 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf04ab1-ce0c-4699-ab82-8de7892f624f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:05.139333 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf04ab1-ce0c-4699-ab82-8de7892f624f admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:05.140009 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf04ab1-ce0c-4699-ab82-8de7892f624f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:05.141685 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf04ab1-ce0c-4699-ab82-8de7892f624f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:05.196441 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f930940c-27f8-4502-acab-c0d2a3d68270 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:05.228799 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 916/1836] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:05 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 01:07:05.229242 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f930940c-27f8-4502-acab-c0d2a3d68270 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:05.252363 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f2222-95c6-4d6f-8b8f-9b7bbe8c476c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:05.253614 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f2222-95c6-4d6f-8b8f-9b7bbe8c476c admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:05.255635 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f2222-95c6-4d6f-8b8f-9b7bbe8c476c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:05.259611 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f2222-95c6-4d6f-8b8f-9b7bbe8c476c admin admin] PATH_INFO: `/v3/projects/e76e4004f076461d8a16cd7b380fb1dc/users/b2e2c37a891245e0ad13258afe7fda1d/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:05.262556 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20f2222-95c6-4d6f-8b8f-9b7bbe8c476c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e76e4004f076461d8a16cd7b380fb1dc, user_id=b2e2c37a891245e0ad13258afe7fda1d, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:05.267086 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20f2222-95c6-4d6f-8b8f-9b7bbe8c476c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:05.306164 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 921/1837] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:05 2022] PUT /identity/v3/projects/e76e4004f076461d8a16cd7b380fb1dc/users/b2e2c37a891245e0ad13258afe7fda1d/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:05.331326 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10941271-8f52-4a55-93e9-fe8fc71d547f None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:05.331896 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10941271-8f52-4a55-93e9-fe8fc71d547f None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:05.331896 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10941271-8f52-4a55-93e9-fe8fc71d547f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:05.403050 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-10941271-8f52-4a55-93e9-fe8fc71d547f None None] MFA Rules not processed for user `b2e2c37a891245e0ad13258afe7fda1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:05.405032 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-10941271-8f52-4a55-93e9-fe8fc71d547f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:05.449768 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:05.570878 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 917/1838] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:05 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:05.583543 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f2222-95c6-4d6f-8b8f-9b7bbe8c476c admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:05.585671 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e20f2222-95c6-4d6f-8b8f-9b7bbe8c476c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:05.753166 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f2222-95c6-4d6f-8b8f-9b7bbe8c476c admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:05.758060 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf4483-6da6-4f20-91e4-06348f31495e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:05.759068 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf4483-6da6-4f20-91e4-06348f31495e service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:05.759718 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf4483-6da6-4f20-91e4-06348f31495e service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:05.760205 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf4483-6da6-4f20-91e4-06348f31495e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:05.777157 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bf4483-6da6-4f20-91e4-06348f31495e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:05.779168 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bf4483-6da6-4f20-91e4-06348f31495e service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:05.794453 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 922/1839] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:05 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:07:05.950375 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10941271-8f52-4a55-93e9-fe8fc71d547f None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:05.966587 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10941271-8f52-4a55-93e9-fe8fc71d547f None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:05.973242 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce837d5-b213-4771-8faf-2c5ff051b8b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:05.979204 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce837d5-b213-4771-8faf-2c5ff051b8b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:05.979204 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce837d5-b213-4771-8faf-2c5ff051b8b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:05.979204 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce837d5-b213-4771-8faf-2c5ff051b8b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:06.067323 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce837d5-b213-4771-8faf-2c5ff051b8b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:06.071819 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce837d5-b213-4771-8faf-2c5ff051b8b0 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:06.096917 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 918/1840] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:05 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:07:06.117403 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf4483-6da6-4f20-91e4-06348f31495e service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:06.271167 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf4483-6da6-4f20-91e4-06348f31495e service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:06.283480 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97f434-283b-41bf-ab0f-0b5870fc1aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:06.284398 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97f434-283b-41bf-ab0f-0b5870fc1aad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:06.284712 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97f434-283b-41bf-ab0f-0b5870fc1aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:06.286899 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97f434-283b-41bf-ab0f-0b5870fc1aad admin admin] PATH_INFO: `/v3/users/b8551a2cd0d042ec83cd1afc8cc721b3` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:06.289565 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be97f434-283b-41bf-ab0f-0b5870fc1aad admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8551a2cd0d042ec83cd1afc8cc721b3)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:06.292046 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be97f434-283b-41bf-ab0f-0b5870fc1aad admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:06.925497 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-be97f434-283b-41bf-ab0f-0b5870fc1aad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8551a2cd0d042ec83cd1afc8cc721b3'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:06.926578 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-be97f434-283b-41bf-ab0f-0b5870fc1aad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8551a2cd0d042ec83cd1afc8cc721b3'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:07.011116 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-be97f434-283b-41bf-ab0f-0b5870fc1aad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8551a2cd0d042ec83cd1afc8cc721b3'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:07.013051 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-be97f434-283b-41bf-ab0f-0b5870fc1aad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8551a2cd0d042ec83cd1afc8cc721b3'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:07.027960 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-be97f434-283b-41bf-ab0f-0b5870fc1aad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8551a2cd0d042ec83cd1afc8cc721b3'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:07.056935 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 923/1841] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:07:06 2022] DELETE /identity/v3/users/b8551a2cd0d042ec83cd1afc8cc721b3 => generated 0 bytes in 941 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:07.063441 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc102a9-c627-41e7-8f2b-632c05c929fd None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:07.064867 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc102a9-c627-41e7-8f2b-632c05c929fd None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:07.065544 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc102a9-c627-41e7-8f2b-632c05c929fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:07.169299 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-efc102a9-c627-41e7-8f2b-632c05c929fd None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:07.169299 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-efc102a9-c627-41e7-8f2b-632c05c929fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:07.169299 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 919/1842] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:07.251181 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97f434-283b-41bf-ab0f-0b5870fc1aad admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:07.254149 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-be97f434-283b-41bf-ab0f-0b5870fc1aad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:07.395190 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97f434-283b-41bf-ab0f-0b5870fc1aad admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:07.400007 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa08997-91ae-48a4-9175-de5f687245da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:07.400904 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa08997-91ae-48a4-9175-de5f687245da service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:07.401694 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa08997-91ae-48a4-9175-de5f687245da service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:07.402449 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa08997-91ae-48a4-9175-de5f687245da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:07.415172 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa08997-91ae-48a4-9175-de5f687245da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:07.424313 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa08997-91ae-48a4-9175-de5f687245da service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:07.445282 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 924/1843] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:07:07.568325 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc102a9-c627-41e7-8f2b-632c05c929fd None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:07.569910 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-efc102a9-c627-41e7-8f2b-632c05c929fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:07.587931 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc102a9-c627-41e7-8f2b-632c05c929fd None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:07.606795 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3bdf9f-ba48-4358-8238-0bb73854adad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:07.611168 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3bdf9f-ba48-4358-8238-0bb73854adad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:07.614066 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3bdf9f-ba48-4358-8238-0bb73854adad admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:07.614066 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3bdf9f-ba48-4358-8238-0bb73854adad admin admin] PATH_INFO: `/v3/users/efba360b4cb042b0aec90295ae1d951a` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:07.617895 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3bdf9f-ba48-4358-8238-0bb73854adad admin admin] RBAC: Authorizing `identity:delete_user(user_id=efba360b4cb042b0aec90295ae1d951a)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:07.621136 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3bdf9f-ba48-4358-8238-0bb73854adad admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:07.953751 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5b3bdf9f-ba48-4358-8238-0bb73854adad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efba360b4cb042b0aec90295ae1d951a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:07.955772 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5b3bdf9f-ba48-4358-8238-0bb73854adad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efba360b4cb042b0aec90295ae1d951a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:07.975891 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5b3bdf9f-ba48-4358-8238-0bb73854adad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efba360b4cb042b0aec90295ae1d951a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:08.057222 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5b3bdf9f-ba48-4358-8238-0bb73854adad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efba360b4cb042b0aec90295ae1d951a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:08.058337 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5b3bdf9f-ba48-4358-8238-0bb73854adad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efba360b4cb042b0aec90295ae1d951a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:08.091645 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 920/1844] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:07:07 2022] DELETE /identity/v3/users/efba360b4cb042b0aec90295ae1d951a => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:08.097183 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70382573-c0fd-4ed8-a2f0-f432c3d7dffb None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:08.098114 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70382573-c0fd-4ed8-a2f0-f432c3d7dffb None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:08.098598 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70382573-c0fd-4ed8-a2f0-f432c3d7dffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:08.186080 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-70382573-c0fd-4ed8-a2f0-f432c3d7dffb None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:08.193799 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-70382573-c0fd-4ed8-a2f0-f432c3d7dffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:08.354697 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 925/1845] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:07:08.375259 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3bdf9f-ba48-4358-8238-0bb73854adad admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:08.378692 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5b3bdf9f-ba48-4358-8238-0bb73854adad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:08.491051 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3bdf9f-ba48-4358-8238-0bb73854adad admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:08.495040 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f905aec0-f02f-4aa5-83e7-ffa5e8a42b69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:08.499626 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905aec0-f02f-4aa5-83e7-ffa5e8a42b69 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:08.499626 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905aec0-f02f-4aa5-83e7-ffa5e8a42b69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:08.499626 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905aec0-f02f-4aa5-83e7-ffa5e8a42b69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:08.506104 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f905aec0-f02f-4aa5-83e7-ffa5e8a42b69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:08.507551 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f905aec0-f02f-4aa5-83e7-ffa5e8a42b69 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:08.518949 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 921/1846] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 01:07:08.828753 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70382573-c0fd-4ed8-a2f0-f432c3d7dffb None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:08.835791 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-70382573-c0fd-4ed8-a2f0-f432c3d7dffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:08.863641 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70382573-c0fd-4ed8-a2f0-f432c3d7dffb None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:08.869823 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e653f-9909-4a93-8afd-71467e80d54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:08.870616 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e653f-9909-4a93-8afd-71467e80d54e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:08.870977 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e653f-9909-4a93-8afd-71467e80d54e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:08.871440 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e653f-9909-4a93-8afd-71467e80d54e admin admin] PATH_INFO: `/v3/projects/e8b6424a4ded4345bfca3379d4a3d718` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:08.873762 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8e653f-9909-4a93-8afd-71467e80d54e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8b6424a4ded4345bfca3379d4a3d718)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:08.884905 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8e653f-9909-4a93-8afd-71467e80d54e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:09.034524 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-3b8e653f-9909-4a93-8afd-71467e80d54e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8b6424a4ded4345bfca3379d4a3d718', 'request_id': 'req-3b8e653f-9909-4a93-8afd-71467e80d54e'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:09.096268 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f905aec0-f02f-4aa5-83e7-ffa5e8a42b69 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:09.096268 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f905aec0-f02f-4aa5-83e7-ffa5e8a42b69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:09.111317 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f905aec0-f02f-4aa5-83e7-ffa5e8a42b69 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:09.115600 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-3b8e653f-9909-4a93-8afd-71467e80d54e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8b6424a4ded4345bfca3379d4a3d718', 'request_id': 'req-3b8e653f-9909-4a93-8afd-71467e80d54e'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:09.123294 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-3b8e653f-9909-4a93-8afd-71467e80d54e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8b6424a4ded4345bfca3379d4a3d718', 'request_id': 'req-3b8e653f-9909-4a93-8afd-71467e80d54e'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:09.135649 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-3b8e653f-9909-4a93-8afd-71467e80d54e admin admin] The token cache is being invalidate because project e8b6424a4ded4345bfca3379d4a3d718 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:07:09.135937 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-3b8e653f-9909-4a93-8afd-71467e80d54e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:09.136623 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-3b8e653f-9909-4a93-8afd-71467e80d54e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:09.138151 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 926/1847] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:07:08 2022] DELETE /identity/v3/projects/e8b6424a4ded4345bfca3379d4a3d718 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:09.142854 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e653f-9909-4a93-8afd-71467e80d54e admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:09.153148 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3b8e653f-9909-4a93-8afd-71467e80d54e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:09.172944 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45628fa4-fb03-490f-8ce7-75d20916c491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:09.174346 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45628fa4-fb03-490f-8ce7-75d20916c491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:09.174714 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45628fa4-fb03-490f-8ce7-75d20916c491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:09.174962 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45628fa4-fb03-490f-8ce7-75d20916c491 admin admin] PATH_INFO: `/v3/projects/8882297dcd374e30bccb1ecb34f7b6c0` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:09.176103 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45628fa4-fb03-490f-8ce7-75d20916c491 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8882297dcd374e30bccb1ecb34f7b6c0)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:09.178070 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45628fa4-fb03-490f-8ce7-75d20916c491 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:09.210299 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e653f-9909-4a93-8afd-71467e80d54e admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:09.215041 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac72467-9b2f-45e0-af16-f539c66d6292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:09.215677 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac72467-9b2f-45e0-af16-f539c66d6292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:09.215962 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac72467-9b2f-45e0-af16-f539c66d6292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:09.216239 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac72467-9b2f-45e0-af16-f539c66d6292 admin admin] PATH_INFO: `/v3/users/6d9a7cf18e7d40678501b1ea47c060af` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:09.217419 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac72467-9b2f-45e0-af16-f539c66d6292 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d9a7cf18e7d40678501b1ea47c060af)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:09.219402 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac72467-9b2f-45e0-af16-f539c66d6292 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:09.311963 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-45628fa4-fb03-490f-8ce7-75d20916c491 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8882297dcd374e30bccb1ecb34f7b6c0', 'request_id': 'req-45628fa4-fb03-490f-8ce7-75d20916c491'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:09.373937 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-45628fa4-fb03-490f-8ce7-75d20916c491 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8882297dcd374e30bccb1ecb34f7b6c0', 'request_id': 'req-45628fa4-fb03-490f-8ce7-75d20916c491'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:09.379928 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-45628fa4-fb03-490f-8ce7-75d20916c491 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8882297dcd374e30bccb1ecb34f7b6c0', 'request_id': 'req-45628fa4-fb03-490f-8ce7-75d20916c491'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:09.391099 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-45628fa4-fb03-490f-8ce7-75d20916c491 admin admin] The token cache is being invalidate because project 8882297dcd374e30bccb1ecb34f7b6c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:07:09.391387 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-45628fa4-fb03-490f-8ce7-75d20916c491 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:09.392150 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-45628fa4-fb03-490f-8ce7-75d20916c491 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:09.393716 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 922/1848] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:07:09 2022] DELETE /identity/v3/projects/8882297dcd374e30bccb1ecb34f7b6c0 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:09.397531 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45628fa4-fb03-490f-8ce7-75d20916c491 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:09.398803 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-45628fa4-fb03-490f-8ce7-75d20916c491 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:09.417595 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-3ac72467-9b2f-45e0-af16-f539c66d6292 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d9a7cf18e7d40678501b1ea47c060af'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:09.419005 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-3ac72467-9b2f-45e0-af16-f539c66d6292 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d9a7cf18e7d40678501b1ea47c060af'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:09.464015 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45628fa4-fb03-490f-8ce7-75d20916c491 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:09.468762 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31dd26d-4b91-446e-9342-d1af88b4b66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:09.469396 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31dd26d-4b91-446e-9342-d1af88b4b66c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:09.469681 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31dd26d-4b91-446e-9342-d1af88b4b66c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:09.470652 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31dd26d-4b91-446e-9342-d1af88b4b66c admin admin] PATH_INFO: `/v3/users/39c101b9c9a14c7db866db9906026f7e` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:09.472006 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31dd26d-4b91-446e-9342-d1af88b4b66c admin admin] RBAC: Authorizing `identity:delete_user(user_id=39c101b9c9a14c7db866db9906026f7e)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:09.474031 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31dd26d-4b91-446e-9342-d1af88b4b66c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:09.508080 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-3ac72467-9b2f-45e0-af16-f539c66d6292 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d9a7cf18e7d40678501b1ea47c060af'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:09.512385 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-3ac72467-9b2f-45e0-af16-f539c66d6292 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d9a7cf18e7d40678501b1ea47c060af'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:09.525105 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-3ac72467-9b2f-45e0-af16-f539c66d6292 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d9a7cf18e7d40678501b1ea47c060af'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:09.542564 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 927/1849] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:07:09 2022] DELETE /identity/v3/users/6d9a7cf18e7d40678501b1ea47c060af => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:07:09.684766 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e31dd26d-4b91-446e-9342-d1af88b4b66c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39c101b9c9a14c7db866db9906026f7e'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:09.685531 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e31dd26d-4b91-446e-9342-d1af88b4b66c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39c101b9c9a14c7db866db9906026f7e'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:09.692590 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e31dd26d-4b91-446e-9342-d1af88b4b66c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39c101b9c9a14c7db866db9906026f7e'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:09.823842 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e31dd26d-4b91-446e-9342-d1af88b4b66c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39c101b9c9a14c7db866db9906026f7e'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:09.823842 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e31dd26d-4b91-446e-9342-d1af88b4b66c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39c101b9c9a14c7db866db9906026f7e'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:09.823842 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:09.857645 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 923/1850] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:07:09 2022] DELETE /identity/v3/users/39c101b9c9a14c7db866db9906026f7e => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:07:10.006026 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:10.168076 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac72467-9b2f-45e0-af16-f539c66d6292 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:10.170415 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3ac72467-9b2f-45e0-af16-f539c66d6292 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:10.329074 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac72467-9b2f-45e0-af16-f539c66d6292 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:10.395518 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752b492b-52da-4307-92e8-8d66e84b4fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:10.396151 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b492b-52da-4307-92e8-8d66e84b4fce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:10.396435 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b492b-52da-4307-92e8-8d66e84b4fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:10.396746 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b492b-52da-4307-92e8-8d66e84b4fce admin admin] PATH_INFO: `/v3/projects/fd38e4cafd2343b696701e1815b10fdd` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:10.397870 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752b492b-52da-4307-92e8-8d66e84b4fce admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd38e4cafd2343b696701e1815b10fdd)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:10.399814 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752b492b-52da-4307-92e8-8d66e84b4fce admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:10.502006 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31dd26d-4b91-446e-9342-d1af88b4b66c admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:10.504161 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e31dd26d-4b91-446e-9342-d1af88b4b66c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:10.529574 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31dd26d-4b91-446e-9342-d1af88b4b66c admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:10.535156 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b30c68-647e-43e8-b963-238cd1f1c71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:10.535785 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b30c68-647e-43e8-b963-238cd1f1c71c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:10.536060 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b30c68-647e-43e8-b963-238cd1f1c71c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:10.536331 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b30c68-647e-43e8-b963-238cd1f1c71c admin admin] PATH_INFO: `/v3/projects/a839cb9e24574e21bff08a1a05fd9f93` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:10.537468 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b30c68-647e-43e8-b963-238cd1f1c71c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a839cb9e24574e21bff08a1a05fd9f93)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:10.539436 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b30c68-647e-43e8-b963-238cd1f1c71c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:10.557891 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-752b492b-52da-4307-92e8-8d66e84b4fce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd38e4cafd2343b696701e1815b10fdd', 'request_id': 'req-752b492b-52da-4307-92e8-8d66e84b4fce'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:10.652767 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-752b492b-52da-4307-92e8-8d66e84b4fce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd38e4cafd2343b696701e1815b10fdd', 'request_id': 'req-752b492b-52da-4307-92e8-8d66e84b4fce'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:10.660799 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-752b492b-52da-4307-92e8-8d66e84b4fce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd38e4cafd2343b696701e1815b10fdd', 'request_id': 'req-752b492b-52da-4307-92e8-8d66e84b4fce'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:10.673747 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-752b492b-52da-4307-92e8-8d66e84b4fce admin admin] The token cache is being invalidate because project fd38e4cafd2343b696701e1815b10fdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:07:10.674323 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-752b492b-52da-4307-92e8-8d66e84b4fce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:10.675149 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-752b492b-52da-4307-92e8-8d66e84b4fce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:10.675935 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-33b30c68-647e-43e8-b963-238cd1f1c71c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a839cb9e24574e21bff08a1a05fd9f93', 'request_id': 'req-33b30c68-647e-43e8-b963-238cd1f1c71c'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:10.677431 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 928/1851] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:07:10 2022] DELETE /identity/v3/projects/fd38e4cafd2343b696701e1815b10fdd => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:10.682481 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1cfcf-5d49-4291-a215-013d2d64afba None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:10.683095 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1cfcf-5d49-4291-a215-013d2d64afba None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:10.683518 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1cfcf-5d49-4291-a215-013d2d64afba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:10.713389 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-8fe1cfcf-5d49-4291-a215-013d2d64afba None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:10.713389 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8fe1cfcf-5d49-4291-a215-013d2d64afba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:10.768717 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-33b30c68-647e-43e8-b963-238cd1f1c71c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a839cb9e24574e21bff08a1a05fd9f93', 'request_id': 'req-33b30c68-647e-43e8-b963-238cd1f1c71c'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:10.777354 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-33b30c68-647e-43e8-b963-238cd1f1c71c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a839cb9e24574e21bff08a1a05fd9f93', 'request_id': 'req-33b30c68-647e-43e8-b963-238cd1f1c71c'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:10.812695 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-33b30c68-647e-43e8-b963-238cd1f1c71c admin admin] The token cache is being invalidate because project a839cb9e24574e21bff08a1a05fd9f93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:07:10.812695 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-33b30c68-647e-43e8-b963-238cd1f1c71c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:10.812695 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-33b30c68-647e-43e8-b963-238cd1f1c71c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:10.812695 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 924/1852] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:07:10 2022] DELETE /identity/v3/projects/a839cb9e24574e21bff08a1a05fd9f93 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:07:10.819654 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacab23c-d6da-409e-bb97-1e99d1ea11a6 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:10.819947 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacab23c-d6da-409e-bb97-1e99d1ea11a6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:10.820228 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacab23c-d6da-409e-bb97-1e99d1ea11a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:10.843977 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 929/1853] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:07:10.846744 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca97ac5-b43f-40a7-8daf-200b8787635b None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:10.846893 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-dacab23c-d6da-409e-bb97-1e99d1ea11a6 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:10.847345 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca97ac5-b43f-40a7-8daf-200b8787635b None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:10.847756 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca97ac5-b43f-40a7-8daf-200b8787635b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:10.848448 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-dacab23c-d6da-409e-bb97-1e99d1ea11a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:10.870407 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 925/1854] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:10.878786 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfa22d-b569-4c5e-8802-3d402008f4b2 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:10.879078 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfa22d-b569-4c5e-8802-3d402008f4b2 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:10.879354 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfa22d-b569-4c5e-8802-3d402008f4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:10.921085 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-cca97ac5-b43f-40a7-8daf-200b8787635b None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:10.922465 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-cca97ac5-b43f-40a7-8daf-200b8787635b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:10.922684 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1adfa22d-b569-4c5e-8802-3d402008f4b2 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:10.924796 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1adfa22d-b569-4c5e-8802-3d402008f4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:10.940007 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 930/1855] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:10.952569 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca97ac5-b43f-40a7-8daf-200b8787635b None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:10.957864 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca97ac5-b43f-40a7-8daf-200b8787635b None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:10.963362 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a375ca8-d5e7-44e7-8432-a30995a31115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:10.963656 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 926/1856] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:10.968134 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a375ca8-d5e7-44e7-8432-a30995a31115 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:10.968134 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a375ca8-d5e7-44e7-8432-a30995a31115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:10.968134 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a375ca8-d5e7-44e7-8432-a30995a31115 admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:10.968134 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a375ca8-d5e7-44e7-8432-a30995a31115 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:07:10.968134 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a375ca8-d5e7-44e7-8432-a30995a31115 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:10.968134 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a375ca8-d5e7-44e7-8432-a30995a31115 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:10.969163 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adfa22d-b569-4c5e-8802-3d402008f4b2 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:10.979427 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adfa22d-b569-4c5e-8802-3d402008f4b2 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:10.984489 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 931/1857] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:07:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 01:07:10.984624 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd9a14b-9c28-4dc1-aa60-ee13eaaff34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:10.984821 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd9a14b-9c28-4dc1-aa60-ee13eaaff34c admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:10.985102 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd9a14b-9c28-4dc1-aa60-ee13eaaff34c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:10.985380 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd9a14b-9c28-4dc1-aa60-ee13eaaff34c admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:10.986557 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd9a14b-9c28-4dc1-aa60-ee13eaaff34c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:07:10.986855 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd9a14b-9c28-4dc1-aa60-ee13eaaff34c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:10.986986 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a375ca8-d5e7-44e7-8432-a30995a31115 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:10.988435 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd9a14b-9c28-4dc1-aa60-ee13eaaff34c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:10.996835 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a375ca8-d5e7-44e7-8432-a30995a31115 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:11.001050 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 927/1858] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:07:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:07:11.002465 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447ff0c2-2b9c-4e35-8f44-9ad03ba065cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:11.003243 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ff0c2-2b9c-4e35-8f44-9ad03ba065cc admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:11.003533 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ff0c2-2b9c-4e35-8f44-9ad03ba065cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:11.003813 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ff0c2-2b9c-4e35-8f44-9ad03ba065cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:11.004452 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447ff0c2-2b9c-4e35-8f44-9ad03ba065cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:11.006034 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd9a14b-9c28-4dc1-aa60-ee13eaaff34c admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:11.006592 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447ff0c2-2b9c-4e35-8f44-9ad03ba065cc admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:11.016499 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd9a14b-9c28-4dc1-aa60-ee13eaaff34c admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:11.022372 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6df232-fa3c-4b02-8706-8361f0f1c97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:11.022979 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6df232-fa3c-4b02-8706-8361f0f1c97c admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:11.023289 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6df232-fa3c-4b02-8706-8361f0f1c97c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:11.023566 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6df232-fa3c-4b02-8706-8361f0f1c97c admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:11.024203 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6df232-fa3c-4b02-8706-8361f0f1c97c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:11.026122 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6df232-fa3c-4b02-8706-8361f0f1c97c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:11.113473 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 932/1859] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:07:10 2022] POST /identity/v3/projects => generated 367 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:11.114709 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447ff0c2-2b9c-4e35-8f44-9ad03ba065cc admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:11.139817 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 928/1860] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:07:11 2022] POST /identity/v3/projects => generated 367 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:07:11.145875 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6df232-fa3c-4b02-8706-8361f0f1c97c admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:11.178948 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447ff0c2-2b9c-4e35-8f44-9ad03ba065cc admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:11.213737 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6df232-fa3c-4b02-8706-8361f0f1c97c admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:11.230246 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2633c-debb-4037-a563-f8ecca06cd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:11.230715 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2633c-debb-4037-a563-f8ecca06cd20 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:11.230896 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2633c-debb-4037-a563-f8ecca06cd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:11.231139 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2633c-debb-4037-a563-f8ecca06cd20 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:11.233956 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2633c-debb-4037-a563-f8ecca06cd20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:11.236169 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2633c-debb-4037-a563-f8ecca06cd20 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:11.253854 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a5982-3346-4eb5-8cca-16e1ea01c723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:11.254426 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a5982-3346-4eb5-8cca-16e1ea01c723 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:11.254695 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a5982-3346-4eb5-8cca-16e1ea01c723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:11.254974 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a5982-3346-4eb5-8cca-16e1ea01c723 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:11.255620 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40a5982-3346-4eb5-8cca-16e1ea01c723 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:11.257641 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40a5982-3346-4eb5-8cca-16e1ea01c723 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:11.346209 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 933/1861] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:07:11 2022] POST /identity/v3/users => generated 284 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:07:11.348066 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 929/1862] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:07:11 2022] POST /identity/v3/users => generated 284 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:07:11.352747 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a5982-3346-4eb5-8cca-16e1ea01c723 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:11.358268 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2633c-debb-4037-a563-f8ecca06cd20 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:11.407816 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2633c-debb-4037-a563-f8ecca06cd20 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:11.408147 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a5982-3346-4eb5-8cca-16e1ea01c723 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:11.413215 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bcf338-b648-409d-84ea-e1a83381ff00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:11.413483 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a50144-6ab4-48c7-ad6c-6b5209299717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:11.413956 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bcf338-b648-409d-84ea-e1a83381ff00 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:11.414085 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a50144-6ab4-48c7-ad6c-6b5209299717 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:11.414284 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a50144-6ab4-48c7-ad6c-6b5209299717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:11.414574 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a50144-6ab4-48c7-ad6c-6b5209299717 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:11.414685 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bcf338-b648-409d-84ea-e1a83381ff00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:11.414883 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bcf338-b648-409d-84ea-e1a83381ff00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:11.415346 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a50144-6ab4-48c7-ad6c-6b5209299717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:11.415763 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bcf338-b648-409d-84ea-e1a83381ff00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:11.416978 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a50144-6ab4-48c7-ad6c-6b5209299717 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:11.417539 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bcf338-b648-409d-84ea-e1a83381ff00 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:11.430975 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 930/1863] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:11 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:11.439498 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 934/1864] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:11 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 01:07:11.439704 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a50144-6ab4-48c7-ad6c-6b5209299717 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:11.439793 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bcf338-b648-409d-84ea-e1a83381ff00 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:11.443024 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bcf338-b648-409d-84ea-e1a83381ff00 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:11.443662 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a50144-6ab4-48c7-ad6c-6b5209299717 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:11.447929 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a408d1b-55b6-4a36-a921-7e4424f504d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:11.448572 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a408d1b-55b6-4a36-a921-7e4424f504d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:11.448858 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a408d1b-55b6-4a36-a921-7e4424f504d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:11.449133 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a408d1b-55b6-4a36-a921-7e4424f504d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:11.449772 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a408d1b-55b6-4a36-a921-7e4424f504d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:11.450427 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a0b1c-2289-4efe-a662-f5c6199f87ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:11.451323 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a0b1c-2289-4efe-a662-f5c6199f87ae admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:11.451576 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a408d1b-55b6-4a36-a921-7e4424f504d7 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:11.451930 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a0b1c-2289-4efe-a662-f5c6199f87ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:11.452351 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a0b1c-2289-4efe-a662-f5c6199f87ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:11.453172 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a0b1c-2289-4efe-a662-f5c6199f87ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:11.455690 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a0b1c-2289-4efe-a662-f5c6199f87ae admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:11.465405 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 935/1865] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:11.468707 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 931/1866] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:11.472647 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a0b1c-2289-4efe-a662-f5c6199f87ae admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:11.472791 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a408d1b-55b6-4a36-a921-7e4424f504d7 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:11.482870 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a408d1b-55b6-4a36-a921-7e4424f504d7 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:11.483280 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a0b1c-2289-4efe-a662-f5c6199f87ae admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:11.489047 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becca766-3708-4ebf-9a0f-2a53f9168ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:11.490433 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becca766-3708-4ebf-9a0f-2a53f9168ad3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:11.490721 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becca766-3708-4ebf-9a0f-2a53f9168ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:11.491008 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becca766-3708-4ebf-9a0f-2a53f9168ad3 admin admin] PATH_INFO: `/v3/projects/bc3ba581f882400dbc65d9a28d0de9fc/users/d76ec7c3115745c9824df8b1823a87c4/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:11.492330 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644226c-23d2-4707-93d3-8429a7edff4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:11.493019 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644226c-23d2-4707-93d3-8429a7edff4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:11.493299 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644226c-23d2-4707-93d3-8429a7edff4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:11.493913 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644226c-23d2-4707-93d3-8429a7edff4a admin admin] PATH_INFO: `/v3/projects/be2a5e6c63be47949b0876b3f810c2e8/users/8d4b094db05e4921b36950e49aa92692/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:11.512710 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becca766-3708-4ebf-9a0f-2a53f9168ad3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc3ba581f882400dbc65d9a28d0de9fc, user_id=d76ec7c3115745c9824df8b1823a87c4, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:11.515719 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becca766-3708-4ebf-9a0f-2a53f9168ad3 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:11.520542 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e644226c-23d2-4707-93d3-8429a7edff4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=be2a5e6c63be47949b0876b3f810c2e8, user_id=8d4b094db05e4921b36950e49aa92692, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:11.523210 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e644226c-23d2-4707-93d3-8429a7edff4a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:11.618855 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 932/1867] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:11 2022] PUT /identity/v3/projects/be2a5e6c63be47949b0876b3f810c2e8/users/8d4b094db05e4921b36950e49aa92692/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:07:11.620810 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 936/1868] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:11 2022] PUT /identity/v3/projects/bc3ba581f882400dbc65d9a28d0de9fc/users/d76ec7c3115745c9824df8b1823a87c4/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:07:11.630028 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4595ee7-433e-453c-96f8-e276c256b29f None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:11.630327 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4595ee7-433e-453c-96f8-e276c256b29f None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:11.630614 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4595ee7-433e-453c-96f8-e276c256b29f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:11.631448 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e8e33-3de1-4dc4-b333-3574fa3f50fe None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:11.631801 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e8e33-3de1-4dc4-b333-3574fa3f50fe None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:11.632172 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e8e33-3de1-4dc4-b333-3574fa3f50fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:11.661876 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-bc0e8e33-3de1-4dc4-b333-3574fa3f50fe None None] MFA Rules not processed for user `d76ec7c3115745c9824df8b1823a87c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:11.662116 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-a4595ee7-433e-453c-96f8-e276c256b29f None None] MFA Rules not processed for user `8d4b094db05e4921b36950e49aa92692`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:11.663118 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a4595ee7-433e-453c-96f8-e276c256b29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:11.663225 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-bc0e8e33-3de1-4dc4-b333-3574fa3f50fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:11.727770 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 933/1869] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:11.731499 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 937/1870] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:11.732908 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4595ee7-433e-453c-96f8-e276c256b29f None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:11.733068 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e8e33-3de1-4dc4-b333-3574fa3f50fe None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:11.803178 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e8e33-3de1-4dc4-b333-3574fa3f50fe None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:11.810269 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4595ee7-433e-453c-96f8-e276c256b29f None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:11.814530 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e84361e-affb-4c88-b6f3-dc772d34603a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:11.814530 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e84361e-affb-4c88-b6f3-dc772d34603a admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:11.814530 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e84361e-affb-4c88-b6f3-dc772d34603a admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:11.814530 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e84361e-affb-4c88-b6f3-dc772d34603a admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:11.814900 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e84361e-affb-4c88-b6f3-dc772d34603a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:11.814900 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e84361e-affb-4c88-b6f3-dc772d34603a admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:11.815857 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88029cc8-708f-4e19-b0cd-f27ec8489b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:11.816573 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88029cc8-708f-4e19-b0cd-f27ec8489b51 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:11.817026 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88029cc8-708f-4e19-b0cd-f27ec8489b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:11.817235 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88029cc8-708f-4e19-b0cd-f27ec8489b51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:11.817938 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88029cc8-708f-4e19-b0cd-f27ec8489b51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:11.819854 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88029cc8-708f-4e19-b0cd-f27ec8489b51 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:11.928676 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 938/1871] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:07:11 2022] POST /identity/v3/projects => generated 365 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:07:11.933223 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e84361e-affb-4c88-b6f3-dc772d34603a admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:11.954576 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 934/1872] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:07:11 2022] POST /identity/v3/projects => generated 367 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:07:11.956093 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88029cc8-708f-4e19-b0cd-f27ec8489b51 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:12.061463 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e84361e-affb-4c88-b6f3-dc772d34603a admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:12.094074 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88029cc8-708f-4e19-b0cd-f27ec8489b51 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:12.099230 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e7d97d-f803-4b9c-8aaa-fa02e8c888de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:12.099852 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7d97d-f803-4b9c-8aaa-fa02e8c888de admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:12.100136 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7d97d-f803-4b9c-8aaa-fa02e8c888de admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:12.100416 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7d97d-f803-4b9c-8aaa-fa02e8c888de admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:12.101091 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e7d97d-f803-4b9c-8aaa-fa02e8c888de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:12.103026 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e7d97d-f803-4b9c-8aaa-fa02e8c888de admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:12.169493 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3db88e-e94f-409e-86e3-98d9694b8b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:12.170116 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3db88e-e94f-409e-86e3-98d9694b8b9f admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:12.170398 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3db88e-e94f-409e-86e3-98d9694b8b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:12.170669 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3db88e-e94f-409e-86e3-98d9694b8b9f admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:12.171303 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3db88e-e94f-409e-86e3-98d9694b8b9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:12.174396 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3db88e-e94f-409e-86e3-98d9694b8b9f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:12.213168 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:12.219581 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 935/1873] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:07:11 2022] POST /identity/v3/users => generated 290 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:12.223924 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e7d97d-f803-4b9c-8aaa-fa02e8c888de admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:12.239433 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e7d97d-f803-4b9c-8aaa-fa02e8c888de admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:12.244200 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff2982-d791-4e30-b55b-c8fccb445ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:12.244841 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff2982-d791-4e30-b55b-c8fccb445ea2 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:12.245169 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff2982-d791-4e30-b55b-c8fccb445ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:12.245545 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff2982-d791-4e30-b55b-c8fccb445ea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:12.246195 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ff2982-d791-4e30-b55b-c8fccb445ea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:12.247773 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ff2982-d791-4e30-b55b-c8fccb445ea2 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:12.260002 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 936/1874] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:12.269400 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff2982-d791-4e30-b55b-c8fccb445ea2 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:12.271317 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 939/1875] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:07:11 2022] POST /identity/v3/users => generated 289 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:07:12.272268 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3db88e-e94f-409e-86e3-98d9694b8b9f admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:12.281562 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff2982-d791-4e30-b55b-c8fccb445ea2 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:12.282291 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3db88e-e94f-409e-86e3-98d9694b8b9f admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:12.287199 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:12.289703 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a88d7-a61e-4e88-82e3-af7fe93b4109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:12.290443 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a88d7-a61e-4e88-82e3-af7fe93b4109 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:12.291036 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a88d7-a61e-4e88-82e3-af7fe93b4109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:12.291132 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a88d7-a61e-4e88-82e3-af7fe93b4109 admin admin] PATH_INFO: `/v3/projects/dcdddc21ff3c40178846a666c72eef3c/users/e546018b2d26451a8ecdda57bfab67a8/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:12.291220 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fec010-6b24-4d2d-b75c-731e03362b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:12.292557 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fec010-6b24-4d2d-b75c-731e03362b9f admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:12.292557 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fec010-6b24-4d2d-b75c-731e03362b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:12.292557 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fec010-6b24-4d2d-b75c-731e03362b9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:12.293072 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fec010-6b24-4d2d-b75c-731e03362b9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:12.294822 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fec010-6b24-4d2d-b75c-731e03362b9f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:12.311385 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 940/1876] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:12.312024 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1a88d7-a61e-4e88-82e3-af7fe93b4109 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcdddc21ff3c40178846a666c72eef3c, user_id=e546018b2d26451a8ecdda57bfab67a8, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:12.312619 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fec010-6b24-4d2d-b75c-731e03362b9f admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:12.315028 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1a88d7-a61e-4e88-82e3-af7fe93b4109 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:12.324101 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fec010-6b24-4d2d-b75c-731e03362b9f admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:12.329674 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56afd6fa-088a-4ca9-8afb-e310d7d478ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:12.330729 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afd6fa-088a-4ca9-8afb-e310d7d478ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:12.331180 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afd6fa-088a-4ca9-8afb-e310d7d478ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:12.331555 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afd6fa-088a-4ca9-8afb-e310d7d478ce admin admin] PATH_INFO: `/v3/projects/9cb9ebee71d14175a203acb2b2251a36/users/450925c27aa14dd4887cba54bab32a75/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:12.347696 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56afd6fa-088a-4ca9-8afb-e310d7d478ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cb9ebee71d14175a203acb2b2251a36, user_id=450925c27aa14dd4887cba54bab32a75, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:12.350434 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56afd6fa-088a-4ca9-8afb-e310d7d478ce admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:12.388431 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 937/1877] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:12 2022] PUT /identity/v3/projects/dcdddc21ff3c40178846a666c72eef3c/users/e546018b2d26451a8ecdda57bfab67a8/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:07:12.392861 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a88d7-a61e-4e88-82e3-af7fe93b4109 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:12.426709 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 941/1878] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:12 2022] PUT /identity/v3/projects/9cb9ebee71d14175a203acb2b2251a36/users/450925c27aa14dd4887cba54bab32a75/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:12.430826 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56afd6fa-088a-4ca9-8afb-e310d7d478ce admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:12.463295 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a88d7-a61e-4e88-82e3-af7fe93b4109 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:12.502029 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56afd6fa-088a-4ca9-8afb-e310d7d478ce admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:12.507391 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b617a-bf6d-476a-af87-b18b3a905923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:12.508012 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b617a-bf6d-476a-af87-b18b3a905923 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:12.509754 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b617a-bf6d-476a-af87-b18b3a905923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:12.509754 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b617a-bf6d-476a-af87-b18b3a905923 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:12.509754 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47b617a-bf6d-476a-af87-b18b3a905923 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:12.511631 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47b617a-bf6d-476a-af87-b18b3a905923 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:12.522764 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 942/1879] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 01:07:12.526953 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b617a-bf6d-476a-af87-b18b3a905923 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:12.539178 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ed147-dd16-44a5-a2d6-f73e7f483fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:12.539178 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ed147-dd16-44a5-a2d6-f73e7f483fab admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:12.539797 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ed147-dd16-44a5-a2d6-f73e7f483fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:12.539884 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b617a-bf6d-476a-af87-b18b3a905923 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:12.539970 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ed147-dd16-44a5-a2d6-f73e7f483fab admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:12.540662 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ed147-dd16-44a5-a2d6-f73e7f483fab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:12.542302 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ed147-dd16-44a5-a2d6-f73e7f483fab admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:12.545066 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d38ece-ab3b-45bc-8d7f-ce7f27e96705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:12.545734 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d38ece-ab3b-45bc-8d7f-ce7f27e96705 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:12.546018 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d38ece-ab3b-45bc-8d7f-ce7f27e96705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:12.546298 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d38ece-ab3b-45bc-8d7f-ce7f27e96705 admin admin] PATH_INFO: `/v3/projects/9cb9ebee71d14175a203acb2b2251a36/users/450925c27aa14dd4887cba54bab32a75/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:12.548864 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d38ece-ab3b-45bc-8d7f-ce7f27e96705 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cb9ebee71d14175a203acb2b2251a36, user_id=450925c27aa14dd4887cba54bab32a75, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:12.551832 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d38ece-ab3b-45bc-8d7f-ce7f27e96705 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:12.553801 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 938/1880] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 01:07:12.557924 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ed147-dd16-44a5-a2d6-f73e7f483fab admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:12.568296 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 943/1881] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:12 2022] PUT /identity/v3/projects/9cb9ebee71d14175a203acb2b2251a36/users/450925c27aa14dd4887cba54bab32a75/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:07:12.577962 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ed147-dd16-44a5-a2d6-f73e7f483fab admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:12.580726 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805b4d0-65f0-4a03-9097-43ccd61c883a None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:12.581205 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805b4d0-65f0-4a03-9097-43ccd61c883a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:12.581659 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805b4d0-65f0-4a03-9097-43ccd61c883a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:12.612767 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-7805b4d0-65f0-4a03-9097-43ccd61c883a None None] MFA Rules not processed for user `450925c27aa14dd4887cba54bab32a75`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:12.614023 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-7805b4d0-65f0-4a03-9097-43ccd61c883a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:12.637486 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1d233-d408-4308-84f8-fb8f4a91b350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:12.638322 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1d233-d408-4308-84f8-fb8f4a91b350 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:12.639038 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1d233-d408-4308-84f8-fb8f4a91b350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:12.639326 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1d233-d408-4308-84f8-fb8f4a91b350 admin admin] PATH_INFO: `/v3/projects/dcdddc21ff3c40178846a666c72eef3c/users/e546018b2d26451a8ecdda57bfab67a8/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:12.641389 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc1d233-d408-4308-84f8-fb8f4a91b350 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcdddc21ff3c40178846a666c72eef3c, user_id=e546018b2d26451a8ecdda57bfab67a8, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:12.645435 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc1d233-d408-4308-84f8-fb8f4a91b350 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:12.660183 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 939/1882] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:12 2022] PUT /identity/v3/projects/dcdddc21ff3c40178846a666c72eef3c/users/e546018b2d26451a8ecdda57bfab67a8/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:07:12.671045 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af0aab-2b49-4a66-bf22-a2540e14b8b5 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:12.671820 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af0aab-2b49-4a66-bf22-a2540e14b8b5 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:12.672255 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af0aab-2b49-4a66-bf22-a2540e14b8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:12.706414 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 944/1883] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:12 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:07:12.710742 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-d7af0aab-2b49-4a66-bf22-a2540e14b8b5 None None] MFA Rules not processed for user `e546018b2d26451a8ecdda57bfab67a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:12.712133 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d7af0aab-2b49-4a66-bf22-a2540e14b8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:12.721627 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805b4d0-65f0-4a03-9097-43ccd61c883a None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:12.723036 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-7805b4d0-65f0-4a03-9097-43ccd61c883a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:12.783015 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805b4d0-65f0-4a03-9097-43ccd61c883a None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:12.788264 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 940/1884] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:12 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:07:12.831981 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a86c0-0561-451c-aeb9-d70994322021 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:12.832549 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a86c0-0561-451c-aeb9-d70994322021 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:12.832839 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a86c0-0561-451c-aeb9-d70994322021 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:12.833113 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a86c0-0561-451c-aeb9-d70994322021 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:12.858024 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:12.893123 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267a86c0-0561-451c-aeb9-d70994322021 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:12.894562 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267a86c0-0561-451c-aeb9-d70994322021 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:12.903797 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 945/1885] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:12 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:07:13.048735 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af0aab-2b49-4a66-bf22-a2540e14b8b5 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:13.058284 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af0aab-2b49-4a66-bf22-a2540e14b8b5 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:13.062279 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56067cf-bcd3-433b-b7d9-93cbddfb8870 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:13.062972 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56067cf-bcd3-433b-b7d9-93cbddfb8870 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:13.063243 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56067cf-bcd3-433b-b7d9-93cbddfb8870 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:13.063527 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56067cf-bcd3-433b-b7d9-93cbddfb8870 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:13.073425 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56067cf-bcd3-433b-b7d9-93cbddfb8870 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:13.074849 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56067cf-bcd3-433b-b7d9-93cbddfb8870 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:13.084540 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 941/1886] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:13 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:07:13.395099 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56067cf-bcd3-433b-b7d9-93cbddfb8870 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:13.396041 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a86c0-0561-451c-aeb9-d70994322021 service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:13.398492 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-267a86c0-0561-451c-aeb9-d70994322021 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:13.399681 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a56067cf-bcd3-433b-b7d9-93cbddfb8870 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:13.421565 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56067cf-bcd3-433b-b7d9-93cbddfb8870 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:13.433113 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f25dd7-6427-4522-9f8c-7c5e1c96f7d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:13.436033 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:13.446239 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f25dd7-6427-4522-9f8c-7c5e1c96f7d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:13.446758 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f25dd7-6427-4522-9f8c-7c5e1c96f7d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:13.447304 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f25dd7-6427-4522-9f8c-7c5e1c96f7d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:13.501091 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a86c0-0561-451c-aeb9-d70994322021 service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:13.505856 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d55676-8b3c-4137-aa19-c86002e5be50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:13.506710 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d55676-8b3c-4137-aa19-c86002e5be50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:13.507116 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d55676-8b3c-4137-aa19-c86002e5be50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:13.507401 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d55676-8b3c-4137-aa19-c86002e5be50 admin admin] PATH_INFO: `/v3/users/3e59e061bb514bd8b3746da7b0a88058` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:13.508758 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d55676-8b3c-4137-aa19-c86002e5be50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e59e061bb514bd8b3746da7b0a88058)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:13.511257 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d55676-8b3c-4137-aa19-c86002e5be50 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:13.512104 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f25dd7-6427-4522-9f8c-7c5e1c96f7d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:13.513829 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f25dd7-6427-4522-9f8c-7c5e1c96f7d8 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:13.524695 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 942/1887] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:13 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:07:13.736919 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-50d55676-8b3c-4137-aa19-c86002e5be50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e59e061bb514bd8b3746da7b0a88058'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:13.738140 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-50d55676-8b3c-4137-aa19-c86002e5be50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e59e061bb514bd8b3746da7b0a88058'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:13.816794 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-50d55676-8b3c-4137-aa19-c86002e5be50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e59e061bb514bd8b3746da7b0a88058'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:13.817491 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-50d55676-8b3c-4137-aa19-c86002e5be50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e59e061bb514bd8b3746da7b0a88058'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:13.830915 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-50d55676-8b3c-4137-aa19-c86002e5be50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e59e061bb514bd8b3746da7b0a88058'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:13.844160 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 946/1888] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:07:13 2022] DELETE /identity/v3/users/3e59e061bb514bd8b3746da7b0a88058 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:13.851769 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1aaa8-e36a-41b1-9429-399b30e073ff None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:13.852725 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1aaa8-e36a-41b1-9429-399b30e073ff None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:13.853159 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1aaa8-e36a-41b1-9429-399b30e073ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:13.875422 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-cfb1aaa8-e36a-41b1-9429-399b30e073ff None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:13.877190 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-cfb1aaa8-e36a-41b1-9429-399b30e073ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:13.936338 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 943/1889] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:07:13.946192 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d55676-8b3c-4137-aa19-c86002e5be50 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:13.947582 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-50d55676-8b3c-4137-aa19-c86002e5be50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:14.009072 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d55676-8b3c-4137-aa19-c86002e5be50 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:14.012813 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d8965-0eb2-4855-a87a-8c501faf7d5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:14.013420 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d8965-0eb2-4855-a87a-8c501faf7d5f service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:14.014714 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d8965-0eb2-4855-a87a-8c501faf7d5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:14.014714 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d8965-0eb2-4855-a87a-8c501faf7d5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:14.024200 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4d8965-0eb2-4855-a87a-8c501faf7d5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:14.025668 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4d8965-0eb2-4855-a87a-8c501faf7d5f service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:14.035889 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 947/1890] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:07:14.037416 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:14.536853 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb1aaa8-e36a-41b1-9429-399b30e073ff None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:14.538454 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-cfb1aaa8-e36a-41b1-9429-399b30e073ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:14.558279 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb1aaa8-e36a-41b1-9429-399b30e073ff None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:14.567073 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4fddb3-d5b4-4c66-b4a4-bafdeec1dbcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:14.567960 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4fddb3-d5b4-4c66-b4a4-bafdeec1dbcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:14.568379 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4fddb3-d5b4-4c66-b4a4-bafdeec1dbcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:14.568719 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4fddb3-d5b4-4c66-b4a4-bafdeec1dbcb admin admin] PATH_INFO: `/v3/projects/66fb024fe3eb4bca99c6ddb79d4e9f37` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:14.570632 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4fddb3-d5b4-4c66-b4a4-bafdeec1dbcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=66fb024fe3eb4bca99c6ddb79d4e9f37)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:14.572720 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4fddb3-d5b4-4c66-b4a4-bafdeec1dbcb admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:14.732619 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:DELETE=15,keystone:SELECT=82,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:14.741582 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-0d4fddb3-d5b4-4c66-b4a4-bafdeec1dbcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66fb024fe3eb4bca99c6ddb79d4e9f37', 'request_id': 'req-0d4fddb3-d5b4-4c66-b4a4-bafdeec1dbcb'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:14.827634 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-0d4fddb3-d5b4-4c66-b4a4-bafdeec1dbcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66fb024fe3eb4bca99c6ddb79d4e9f37', 'request_id': 'req-0d4fddb3-d5b4-4c66-b4a4-bafdeec1dbcb'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:14.833686 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-0d4fddb3-d5b4-4c66-b4a4-bafdeec1dbcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66fb024fe3eb4bca99c6ddb79d4e9f37', 'request_id': 'req-0d4fddb3-d5b4-4c66-b4a4-bafdeec1dbcb'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:14.845533 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-0d4fddb3-d5b4-4c66-b4a4-bafdeec1dbcb admin admin] The token cache is being invalidate because project 66fb024fe3eb4bca99c6ddb79d4e9f37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:07:14.845824 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-0d4fddb3-d5b4-4c66-b4a4-bafdeec1dbcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:14.846495 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-0d4fddb3-d5b4-4c66-b4a4-bafdeec1dbcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:14.848001 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 944/1891] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:07:14 2022] DELETE /identity/v3/projects/66fb024fe3eb4bca99c6ddb79d4e9f37 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:14.852032 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d8965-0eb2-4855-a87a-8c501faf7d5f service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:14.862446 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d8965-0eb2-4855-a87a-8c501faf7d5f service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:14.921742 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3218cbc8-e318-4093-9983-e9bf6fffa4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:14.924049 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3218cbc8-e318-4093-9983-e9bf6fffa4fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:14.924049 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3218cbc8-e318-4093-9983-e9bf6fffa4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:14.924049 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3218cbc8-e318-4093-9983-e9bf6fffa4fe admin admin] PATH_INFO: `/v3/users/b2e2c37a891245e0ad13258afe7fda1d` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:14.924802 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3218cbc8-e318-4093-9983-e9bf6fffa4fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2e2c37a891245e0ad13258afe7fda1d)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:14.926701 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3218cbc8-e318-4093-9983-e9bf6fffa4fe admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:15.121580 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-3218cbc8-e318-4093-9983-e9bf6fffa4fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2e2c37a891245e0ad13258afe7fda1d'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:15.122240 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-3218cbc8-e318-4093-9983-e9bf6fffa4fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2e2c37a891245e0ad13258afe7fda1d'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:15.214439 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-3218cbc8-e318-4093-9983-e9bf6fffa4fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2e2c37a891245e0ad13258afe7fda1d'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:15.215165 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-3218cbc8-e318-4093-9983-e9bf6fffa4fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2e2c37a891245e0ad13258afe7fda1d'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:15.230010 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-3218cbc8-e318-4093-9983-e9bf6fffa4fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2e2c37a891245e0ad13258afe7fda1d'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:15.245405 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 948/1892] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:07:14 2022] DELETE /identity/v3/users/b2e2c37a891245e0ad13258afe7fda1d => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:15.787780 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4fddb3-d5b4-4c66-b4a4-bafdeec1dbcb admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:15.789834 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0d4fddb3-d5b4-4c66-b4a4-bafdeec1dbcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:15.808878 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4fddb3-d5b4-4c66-b4a4-bafdeec1dbcb admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:15.914084 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f721e-daf8-470f-b3c6-7c9ac75bcd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:15.914750 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f721e-daf8-470f-b3c6-7c9ac75bcd94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:15.915099 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f721e-daf8-470f-b3c6-7c9ac75bcd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:15.915303 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f721e-daf8-470f-b3c6-7c9ac75bcd94 admin admin] PATH_INFO: `/v3/projects/e76e4004f076461d8a16cd7b380fb1dc` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:15.916765 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0f721e-daf8-470f-b3c6-7c9ac75bcd94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e76e4004f076461d8a16cd7b380fb1dc)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:15.919544 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0f721e-daf8-470f-b3c6-7c9ac75bcd94 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:16.067020 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9f0f721e-daf8-470f-b3c6-7c9ac75bcd94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e76e4004f076461d8a16cd7b380fb1dc', 'request_id': 'req-9f0f721e-daf8-470f-b3c6-7c9ac75bcd94'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:16.134374 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9f0f721e-daf8-470f-b3c6-7c9ac75bcd94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e76e4004f076461d8a16cd7b380fb1dc', 'request_id': 'req-9f0f721e-daf8-470f-b3c6-7c9ac75bcd94'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:16.143260 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9f0f721e-daf8-470f-b3c6-7c9ac75bcd94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e76e4004f076461d8a16cd7b380fb1dc', 'request_id': 'req-9f0f721e-daf8-470f-b3c6-7c9ac75bcd94'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:16.155068 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9f0f721e-daf8-470f-b3c6-7c9ac75bcd94 admin admin] The token cache is being invalidate because project e76e4004f076461d8a16cd7b380fb1dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:07:16.155368 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9f0f721e-daf8-470f-b3c6-7c9ac75bcd94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:16.156043 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9f0f721e-daf8-470f-b3c6-7c9ac75bcd94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:16.164391 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 945/1893] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:07:15 2022] DELETE /identity/v3/projects/e76e4004f076461d8a16cd7b380fb1dc => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:16.165930 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:16.176895 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94321a-88a8-41ca-be89-dfad4ae342e2 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:16.176895 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94321a-88a8-41ca-be89-dfad4ae342e2 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:16.176895 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94321a-88a8-41ca-be89-dfad4ae342e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:16.208889 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-ee94321a-88a8-41ca-be89-dfad4ae342e2 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:16.208889 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ee94321a-88a8-41ca-be89-dfad4ae342e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:16.276471 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 949/1894] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:16.284091 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc46db-29c6-4ae1-a4a1-fb9db7e0daa1 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:16.287604 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc46db-29c6-4ae1-a4a1-fb9db7e0daa1 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:16.287604 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc46db-29c6-4ae1-a4a1-fb9db7e0daa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:16.306188 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5dbc46db-29c6-4ae1-a4a1-fb9db7e0daa1 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:16.308813 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5dbc46db-29c6-4ae1-a4a1-fb9db7e0daa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:16.316335 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 946/1895] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:16.320404 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee94321a-88a8-41ca-be89-dfad4ae342e2 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:16.330218 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee94321a-88a8-41ca-be89-dfad4ae342e2 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:16.334844 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e89ea5-a1a1-4eb4-9a2f-7c6deb9d4841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:16.338236 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e89ea5-a1a1-4eb4-9a2f-7c6deb9d4841 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:16.338236 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e89ea5-a1a1-4eb4-9a2f-7c6deb9d4841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:16.338236 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e89ea5-a1a1-4eb4-9a2f-7c6deb9d4841 admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:16.338236 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e89ea5-a1a1-4eb4-9a2f-7c6deb9d4841 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:07:16.338236 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e89ea5-a1a1-4eb4-9a2f-7c6deb9d4841 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:16.338670 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e89ea5-a1a1-4eb4-9a2f-7c6deb9d4841 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:16.351019 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 950/1896] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:07:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:07:16.355547 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc46db-29c6-4ae1-a4a1-fb9db7e0daa1 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:16.368103 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc46db-29c6-4ae1-a4a1-fb9db7e0daa1 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:16.370537 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07abae54-adb5-4f16-965e-dd08fe523303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:16.371147 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07abae54-adb5-4f16-965e-dd08fe523303 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:16.371439 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07abae54-adb5-4f16-965e-dd08fe523303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:16.371733 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07abae54-adb5-4f16-965e-dd08fe523303 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:16.372370 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07abae54-adb5-4f16-965e-dd08fe523303 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:16.374307 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07abae54-adb5-4f16-965e-dd08fe523303 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:16.608920 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 947/1897] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:07:16 2022] POST /identity/v3/projects => generated 389 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:16.613034 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e89ea5-a1a1-4eb4-9a2f-7c6deb9d4841 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:16.624989 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e89ea5-a1a1-4eb4-9a2f-7c6deb9d4841 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:16.683610 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e15a8-c75b-4465-80ab-8adc18a2b1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:16.684211 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e15a8-c75b-4465-80ab-8adc18a2b1c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:16.684524 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e15a8-c75b-4465-80ab-8adc18a2b1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:16.684801 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e15a8-c75b-4465-80ab-8adc18a2b1c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:16.685422 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e15a8-c75b-4465-80ab-8adc18a2b1c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:16.687364 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e15a8-c75b-4465-80ab-8adc18a2b1c1 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:16.825505 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 951/1898] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:07:16 2022] POST /identity/v3/users => generated 295 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:16.830329 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07abae54-adb5-4f16-965e-dd08fe523303 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:16.840647 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07abae54-adb5-4f16-965e-dd08fe523303 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:16.845279 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf538011-5b16-456e-8613-b8b548eea7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:16.845984 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf538011-5b16-456e-8613-b8b548eea7b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:16.846452 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf538011-5b16-456e-8613-b8b548eea7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:16.846730 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf538011-5b16-456e-8613-b8b548eea7b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:16.847371 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf538011-5b16-456e-8613-b8b548eea7b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:16.849036 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf538011-5b16-456e-8613-b8b548eea7b4 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:16.860783 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 948/1899] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:16.877236 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e15a8-c75b-4465-80ab-8adc18a2b1c1 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:16.887031 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf538011-5b16-456e-8613-b8b548eea7b4 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:16.889390 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bf538011-5b16-456e-8613-b8b548eea7b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:16.894626 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e15a8-c75b-4465-80ab-8adc18a2b1c1 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:16.902604 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11645565-674f-4b5f-90f8-b0286df89572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:16.903063 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11645565-674f-4b5f-90f8-b0286df89572 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:16.903531 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11645565-674f-4b5f-90f8-b0286df89572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:16.903972 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11645565-674f-4b5f-90f8-b0286df89572 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:16.904873 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11645565-674f-4b5f-90f8-b0286df89572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:16.906660 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11645565-674f-4b5f-90f8-b0286df89572 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:16.908415 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf538011-5b16-456e-8613-b8b548eea7b4 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:16.915582 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df45d0-0a28-409d-b34e-a99289184d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:16.916395 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df45d0-0a28-409d-b34e-a99289184d9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:16.916719 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df45d0-0a28-409d-b34e-a99289184d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:16.917232 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df45d0-0a28-409d-b34e-a99289184d9a admin admin] PATH_INFO: `/v3/users/8d4b094db05e4921b36950e49aa92692` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:16.919267 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01df45d0-0a28-409d-b34e-a99289184d9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d4b094db05e4921b36950e49aa92692)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:16.919579 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 952/1900] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:16.921569 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01df45d0-0a28-409d-b34e-a99289184d9a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:16.923903 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11645565-674f-4b5f-90f8-b0286df89572 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:16.934675 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11645565-674f-4b5f-90f8-b0286df89572 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:16.943663 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f5924-5e7e-4521-8fda-60f05b9ce34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:16.944419 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f5924-5e7e-4521-8fda-60f05b9ce34c admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:16.944805 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f5924-5e7e-4521-8fda-60f05b9ce34c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:16.945091 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f5924-5e7e-4521-8fda-60f05b9ce34c admin admin] PATH_INFO: `/v3/projects/baf79e5942dc4005a95062175e0178ea/users/99cd655140a04f9197cb2197dadfb52f/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:16.951529 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:16.992810 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1f5924-5e7e-4521-8fda-60f05b9ce34c admin admin] RBAC: Authorizing `identity:create_grant(project_id=baf79e5942dc4005a95062175e0178ea, user_id=99cd655140a04f9197cb2197dadfb52f, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:16.997603 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1f5924-5e7e-4521-8fda-60f05b9ce34c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:17.122423 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 953/1901] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:16 2022] PUT /identity/v3/projects/baf79e5942dc4005a95062175e0178ea/users/99cd655140a04f9197cb2197dadfb52f/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:17.141968 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28761a-6b67-41a3-b24b-f8ca8a6ce775 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:17.143888 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28761a-6b67-41a3-b24b-f8ca8a6ce775 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:17.144163 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28761a-6b67-41a3-b24b-f8ca8a6ce775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:17.152424 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-01df45d0-0a28-409d-b34e-a99289184d9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d4b094db05e4921b36950e49aa92692'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:17.153217 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-01df45d0-0a28-409d-b34e-a99289184d9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d4b094db05e4921b36950e49aa92692'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:17.162452 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-01df45d0-0a28-409d-b34e-a99289184d9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d4b094db05e4921b36950e49aa92692'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:17.180595 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-5f28761a-6b67-41a3-b24b-f8ca8a6ce775 None None] MFA Rules not processed for user `99cd655140a04f9197cb2197dadfb52f`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:17.182020 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5f28761a-6b67-41a3-b24b-f8ca8a6ce775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:17.233916 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-01df45d0-0a28-409d-b34e-a99289184d9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d4b094db05e4921b36950e49aa92692'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:17.234702 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-01df45d0-0a28-409d-b34e-a99289184d9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d4b094db05e4921b36950e49aa92692'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:17.255443 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 949/1902] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:07:16 2022] DELETE /identity/v3/users/8d4b094db05e4921b36950e49aa92692 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:07:17.259867 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b445b-15ce-4428-a2d0-fa3b5b54cd9c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:17.260167 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b445b-15ce-4428-a2d0-fa3b5b54cd9c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:17.260510 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b445b-15ce-4428-a2d0-fa3b5b54cd9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:17.260751 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 954/1903] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:17 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:17.262248 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f28761a-6b67-41a3-b24b-f8ca8a6ce775 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:17.263963 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5f28761a-6b67-41a3-b24b-f8ca8a6ce775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:17.290239 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-257b445b-15ce-4428-a2d0-fa3b5b54cd9c None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:17.293510 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-257b445b-15ce-4428-a2d0-fa3b5b54cd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:17.357874 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f28761a-6b67-41a3-b24b-f8ca8a6ce775 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:17.373757 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 950/1904] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:07:17.384680 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b445b-15ce-4428-a2d0-fa3b5b54cd9c None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:17.386734 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-257b445b-15ce-4428-a2d0-fa3b5b54cd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:17.439182 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7ccee-1284-4a13-8bd9-9a483df0b285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:17.440121 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7ccee-1284-4a13-8bd9-9a483df0b285 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:17.440753 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7ccee-1284-4a13-8bd9-9a483df0b285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:17.441600 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7ccee-1284-4a13-8bd9-9a483df0b285 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:17.442467 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b7ccee-1284-4a13-8bd9-9a483df0b285 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:17.444696 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b7ccee-1284-4a13-8bd9-9a483df0b285 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:17.472617 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b445b-15ce-4428-a2d0-fa3b5b54cd9c None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:17.476383 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b375c-92e0-4130-b038-cbbbcf9bec70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:17.477072 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b375c-92e0-4130-b038-cbbbcf9bec70 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:17.477701 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b375c-92e0-4130-b038-cbbbcf9bec70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:17.478001 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b375c-92e0-4130-b038-cbbbcf9bec70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:17.488322 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b375c-92e0-4130-b038-cbbbcf9bec70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:17.490133 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b375c-92e0-4130-b038-cbbbcf9bec70 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:17.500419 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 951/1905] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:07:17.570146 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 955/1906] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:07:17 2022] POST /identity/v3/projects => generated 387 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:17.574191 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b375c-92e0-4130-b038-cbbbcf9bec70 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:17.584268 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b375c-92e0-4130-b038-cbbbcf9bec70 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:17.641627 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee8ec54-5684-43ae-b1fe-e1eff2e99ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:17.642215 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8ec54-5684-43ae-b1fe-e1eff2e99ea3 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:17.642730 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8ec54-5684-43ae-b1fe-e1eff2e99ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:17.643049 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8ec54-5684-43ae-b1fe-e1eff2e99ea3 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:17.643728 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee8ec54-5684-43ae-b1fe-e1eff2e99ea3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:17.645749 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee8ec54-5684-43ae-b1fe-e1eff2e99ea3 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:17.748997 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 952/1907] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:07:17 2022] POST /identity/v3/users => generated 300 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:17.753143 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7ccee-1284-4a13-8bd9-9a483df0b285 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:17.764136 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7ccee-1284-4a13-8bd9-9a483df0b285 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:17.766740 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:17.771969 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc85da7f-7519-4bd0-9d81-ac030efac477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:17.775367 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85da7f-7519-4bd0-9d81-ac030efac477 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:17.775367 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85da7f-7519-4bd0-9d81-ac030efac477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:17.775367 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85da7f-7519-4bd0-9d81-ac030efac477 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:17.775367 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc85da7f-7519-4bd0-9d81-ac030efac477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:17.775740 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc85da7f-7519-4bd0-9d81-ac030efac477 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:17.790418 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 956/1908] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:17.791870 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee8ec54-5684-43ae-b1fe-e1eff2e99ea3 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:17.803068 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee8ec54-5684-43ae-b1fe-e1eff2e99ea3 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:17.810109 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e98571-8a31-4a44-92b0-25b10c3fd613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:17.810779 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e98571-8a31-4a44-92b0-25b10c3fd613 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:17.811064 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e98571-8a31-4a44-92b0-25b10c3fd613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:17.811824 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e98571-8a31-4a44-92b0-25b10c3fd613 admin admin] PATH_INFO: `/v3/projects/6296c1ca3139499f85797dc5f7120cf0/users/74e72860d6a64b7ab521a297ef8359d0/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:17.826818 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e98571-8a31-4a44-92b0-25b10c3fd613 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6296c1ca3139499f85797dc5f7120cf0, user_id=74e72860d6a64b7ab521a297ef8359d0, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:17.830087 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e98571-8a31-4a44-92b0-25b10c3fd613 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:17.908910 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 953/1909] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:17 2022] PUT /identity/v3/projects/6296c1ca3139499f85797dc5f7120cf0/users/74e72860d6a64b7ab521a297ef8359d0/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:17.910137 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc85da7f-7519-4bd0-9d81-ac030efac477 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:17.912740 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e98571-8a31-4a44-92b0-25b10c3fd613 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:17.913509 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-cc85da7f-7519-4bd0-9d81-ac030efac477 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:17.936923 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc85da7f-7519-4bd0-9d81-ac030efac477 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:17.977603 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e98571-8a31-4a44-92b0-25b10c3fd613 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:17.982437 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86568b9-6dd3-407a-a71c-e404ae0b3b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:17.983044 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86568b9-6dd3-407a-a71c-e404ae0b3b90 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:17.983323 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86568b9-6dd3-407a-a71c-e404ae0b3b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:17.983601 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86568b9-6dd3-407a-a71c-e404ae0b3b90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:17.984241 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86568b9-6dd3-407a-a71c-e404ae0b3b90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:17.986050 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86568b9-6dd3-407a-a71c-e404ae0b3b90 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:17.993599 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:17.998640 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aabf022-4d22-4c1e-96fc-9fb0244f19a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:17.999065 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 954/1910] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 01:07:18.000745 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aabf022-4d22-4c1e-96fc-9fb0244f19a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:18.001200 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aabf022-4d22-4c1e-96fc-9fb0244f19a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:18.001498 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aabf022-4d22-4c1e-96fc-9fb0244f19a9 admin admin] PATH_INFO: `/v3/projects/be2a5e6c63be47949b0876b3f810c2e8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:18.002648 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aabf022-4d22-4c1e-96fc-9fb0244f19a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be2a5e6c63be47949b0876b3f810c2e8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:18.003242 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86568b9-6dd3-407a-a71c-e404ae0b3b90 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:18.004736 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aabf022-4d22-4c1e-96fc-9fb0244f19a9 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:18.013151 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86568b9-6dd3-407a-a71c-e404ae0b3b90 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:18.018827 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1130b0-63c8-435a-985d-fe0e3512be1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:18.019466 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1130b0-63c8-435a-985d-fe0e3512be1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:18.019745 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1130b0-63c8-435a-985d-fe0e3512be1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:18.020017 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1130b0-63c8-435a-985d-fe0e3512be1c admin admin] PATH_INFO: `/v3/projects/6296c1ca3139499f85797dc5f7120cf0/users/74e72860d6a64b7ab521a297ef8359d0/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:18.021934 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1130b0-63c8-435a-985d-fe0e3512be1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6296c1ca3139499f85797dc5f7120cf0, user_id=74e72860d6a64b7ab521a297ef8359d0, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:18.024588 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1130b0-63c8-435a-985d-fe0e3512be1c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:18.035854 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 955/1911] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:18 2022] PUT /identity/v3/projects/6296c1ca3139499f85797dc5f7120cf0/users/74e72860d6a64b7ab521a297ef8359d0/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:07:18.047457 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad136a-ca08-4524-806c-338e9bd18d13 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:18.047740 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad136a-ca08-4524-806c-338e9bd18d13 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:18.048014 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad136a-ca08-4524-806c-338e9bd18d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:18.077168 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-a7ad136a-ca08-4524-806c-338e9bd18d13 None None] MFA Rules not processed for user `74e72860d6a64b7ab521a297ef8359d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:18.081368 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a7ad136a-ca08-4524-806c-338e9bd18d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:18.119097 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-5aabf022-4d22-4c1e-96fc-9fb0244f19a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be2a5e6c63be47949b0876b3f810c2e8', 'request_id': 'req-5aabf022-4d22-4c1e-96fc-9fb0244f19a9'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:18.165192 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 956/1912] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:18 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:07:18.214344 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-5aabf022-4d22-4c1e-96fc-9fb0244f19a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be2a5e6c63be47949b0876b3f810c2e8', 'request_id': 'req-5aabf022-4d22-4c1e-96fc-9fb0244f19a9'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:18.219340 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad136a-ca08-4524-806c-338e9bd18d13 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:18.222969 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a7ad136a-ca08-4524-806c-338e9bd18d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:18.231967 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-5aabf022-4d22-4c1e-96fc-9fb0244f19a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be2a5e6c63be47949b0876b3f810c2e8', 'request_id': 'req-5aabf022-4d22-4c1e-96fc-9fb0244f19a9'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:18.251874 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-5aabf022-4d22-4c1e-96fc-9fb0244f19a9 admin admin] The token cache is being invalidate because project be2a5e6c63be47949b0876b3f810c2e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:07:18.256128 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-5aabf022-4d22-4c1e-96fc-9fb0244f19a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:18.256128 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-5aabf022-4d22-4c1e-96fc-9fb0244f19a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:18.260195 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 957/1913] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:07:17 2022] DELETE /identity/v3/projects/be2a5e6c63be47949b0876b3f810c2e8 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:18.261311 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aabf022-4d22-4c1e-96fc-9fb0244f19a9 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:18.265493 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5aabf022-4d22-4c1e-96fc-9fb0244f19a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:18.340901 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad136a-ca08-4524-806c-338e9bd18d13 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:18.424259 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aabf022-4d22-4c1e-96fc-9fb0244f19a9 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:18.430432 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f259f795-5cd3-4156-988c-b4ff74e01827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:18.431140 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259f795-5cd3-4156-988c-b4ff74e01827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:18.431474 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259f795-5cd3-4156-988c-b4ff74e01827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:18.431861 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259f795-5cd3-4156-988c-b4ff74e01827 admin admin] PATH_INFO: `/v3/users/e546018b2d26451a8ecdda57bfab67a8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:18.433054 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f259f795-5cd3-4156-988c-b4ff74e01827 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e546018b2d26451a8ecdda57bfab67a8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:18.435467 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f259f795-5cd3-4156-988c-b4ff74e01827 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:18.444300 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658a39b-6ff9-41b8-81f5-f6bdb60696ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:18.447660 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658a39b-6ff9-41b8-81f5-f6bdb60696ba service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:18.447946 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658a39b-6ff9-41b8-81f5-f6bdb60696ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:18.449037 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658a39b-6ff9-41b8-81f5-f6bdb60696ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:18.450199 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e658a39b-6ff9-41b8-81f5-f6bdb60696ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:18.579039 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e658a39b-6ff9-41b8-81f5-f6bdb60696ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:18.581476 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e658a39b-6ff9-41b8-81f5-f6bdb60696ba service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:18.598672 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 957/1914] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:18 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 01:07:18.703965 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f259f795-5cd3-4156-988c-b4ff74e01827 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e546018b2d26451a8ecdda57bfab67a8'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:18.705148 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f259f795-5cd3-4156-988c-b4ff74e01827 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e546018b2d26451a8ecdda57bfab67a8'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:18.763260 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f259f795-5cd3-4156-988c-b4ff74e01827 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e546018b2d26451a8ecdda57bfab67a8'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:18.764282 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f259f795-5cd3-4156-988c-b4ff74e01827 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e546018b2d26451a8ecdda57bfab67a8'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:18.784689 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f259f795-5cd3-4156-988c-b4ff74e01827 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e546018b2d26451a8ecdda57bfab67a8'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:18.804889 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 958/1915] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:07:18 2022] DELETE /identity/v3/users/e546018b2d26451a8ecdda57bfab67a8 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:18.820643 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:18.972804 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658a39b-6ff9-41b8-81f5-f6bdb60696ba service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:18.974238 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e658a39b-6ff9-41b8-81f5-f6bdb60696ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:18.987221 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658a39b-6ff9-41b8-81f5-f6bdb60696ba service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:19.027428 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f259f795-5cd3-4156-988c-b4ff74e01827 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:19.029277 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f259f795-5cd3-4156-988c-b4ff74e01827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:19.069638 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7741d7d-327c-4aee-a074-ee666807f479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:19.070422 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7741d7d-327c-4aee-a074-ee666807f479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:19.070762 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7741d7d-327c-4aee-a074-ee666807f479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:19.071052 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7741d7d-327c-4aee-a074-ee666807f479 admin admin] PATH_INFO: `/v3/users/99cd655140a04f9197cb2197dadfb52f` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:19.072323 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7741d7d-327c-4aee-a074-ee666807f479 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99cd655140a04f9197cb2197dadfb52f)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:19.074576 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7741d7d-327c-4aee-a074-ee666807f479 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:19.104283 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f259f795-5cd3-4156-988c-b4ff74e01827 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:19.116520 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d960e0-7a6d-4865-b240-56610dc17b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:19.116520 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d960e0-7a6d-4865-b240-56610dc17b7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:19.116520 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d960e0-7a6d-4865-b240-56610dc17b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:19.116520 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d960e0-7a6d-4865-b240-56610dc17b7e admin admin] PATH_INFO: `/v3/users/c7c14f3183254b19aaf33b966698567a` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:19.117764 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d960e0-7a6d-4865-b240-56610dc17b7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7c14f3183254b19aaf33b966698567a)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:19.119840 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d960e0-7a6d-4865-b240-56610dc17b7e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:19.262521 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b7741d7d-327c-4aee-a074-ee666807f479 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99cd655140a04f9197cb2197dadfb52f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:19.263187 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b7741d7d-327c-4aee-a074-ee666807f479 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99cd655140a04f9197cb2197dadfb52f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:19.270292 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b7741d7d-327c-4aee-a074-ee666807f479 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99cd655140a04f9197cb2197dadfb52f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:19.290701 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-05d960e0-7a6d-4865-b240-56610dc17b7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7c14f3183254b19aaf33b966698567a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:19.291594 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-05d960e0-7a6d-4865-b240-56610dc17b7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7c14f3183254b19aaf33b966698567a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:19.368747 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b7741d7d-327c-4aee-a074-ee666807f479 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99cd655140a04f9197cb2197dadfb52f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:19.369540 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b7741d7d-327c-4aee-a074-ee666807f479 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99cd655140a04f9197cb2197dadfb52f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:19.392412 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 958/1916] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:07:18 2022] DELETE /identity/v3/users/99cd655140a04f9197cb2197dadfb52f => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:19.396340 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7741d7d-327c-4aee-a074-ee666807f479 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:19.397768 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b7741d7d-327c-4aee-a074-ee666807f479 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:19.433826 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-05d960e0-7a6d-4865-b240-56610dc17b7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7c14f3183254b19aaf33b966698567a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:19.438766 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-05d960e0-7a6d-4865-b240-56610dc17b7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7c14f3183254b19aaf33b966698567a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:19.453965 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-05d960e0-7a6d-4865-b240-56610dc17b7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7c14f3183254b19aaf33b966698567a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:19.471739 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 959/1917] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:07:19 2022] DELETE /identity/v3/users/c7c14f3183254b19aaf33b966698567a => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:07:19.473754 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe9a80-6769-40aa-abe5-3c21f71e0698 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:19.474625 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe9a80-6769-40aa-abe5-3c21f71e0698 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:19.477636 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe9a80-6769-40aa-abe5-3c21f71e0698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:19.489113 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7741d7d-327c-4aee-a074-ee666807f479 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:19.502255 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-20fe9a80-6769-40aa-abe5-3c21f71e0698 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:19.503604 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-20fe9a80-6769-40aa-abe5-3c21f71e0698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:19.568307 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2e3c7-a172-4c97-b049-ed2c47a77100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:19.569760 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2e3c7-a172-4c97-b049-ed2c47a77100 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:19.570049 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2e3c7-a172-4c97-b049-ed2c47a77100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:19.570452 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2e3c7-a172-4c97-b049-ed2c47a77100 admin admin] PATH_INFO: `/v3/projects/dcdddc21ff3c40178846a666c72eef3c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:19.571904 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a2e3c7-a172-4c97-b049-ed2c47a77100 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcdddc21ff3c40178846a666c72eef3c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:19.574390 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a2e3c7-a172-4c97-b049-ed2c47a77100 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:19.594143 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 960/1918] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:19.596186 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee4778-b009-4c1a-9e64-ee88a60f60e6 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:19.596674 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee4778-b009-4c1a-9e64-ee88a60f60e6 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:19.596947 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee4778-b009-4c1a-9e64-ee88a60f60e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:19.618704 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-88ee4778-b009-4c1a-9e64-ee88a60f60e6 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:19.620034 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-88ee4778-b009-4c1a-9e64-ee88a60f60e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:19.631805 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 961/1919] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:19.633580 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee4778-b009-4c1a-9e64-ee88a60f60e6 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:19.635089 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-88ee4778-b009-4c1a-9e64-ee88a60f60e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:19.740298 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d6a2e3c7-a172-4c97-b049-ed2c47a77100 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcdddc21ff3c40178846a666c72eef3c', 'request_id': 'req-d6a2e3c7-a172-4c97-b049-ed2c47a77100'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:19.752952 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee4778-b009-4c1a-9e64-ee88a60f60e6 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:19.839110 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb4beac-c413-4d94-b454-149a3aab73f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:19.839895 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4beac-c413-4d94-b454-149a3aab73f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:19.840286 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4beac-c413-4d94-b454-149a3aab73f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:19.840698 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4beac-c413-4d94-b454-149a3aab73f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:19.856807 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d6a2e3c7-a172-4c97-b049-ed2c47a77100 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcdddc21ff3c40178846a666c72eef3c', 'request_id': 'req-d6a2e3c7-a172-4c97-b049-ed2c47a77100'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:19.868974 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d6a2e3c7-a172-4c97-b049-ed2c47a77100 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcdddc21ff3c40178846a666c72eef3c', 'request_id': 'req-d6a2e3c7-a172-4c97-b049-ed2c47a77100'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:19.889337 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:19.894520 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d6a2e3c7-a172-4c97-b049-ed2c47a77100 admin admin] The token cache is being invalidate because project dcdddc21ff3c40178846a666c72eef3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:07:19.894520 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d6a2e3c7-a172-4c97-b049-ed2c47a77100 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:19.894520 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d6a2e3c7-a172-4c97-b049-ed2c47a77100 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:19.898005 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 959/1920] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:07:19 2022] DELETE /identity/v3/projects/dcdddc21ff3c40178846a666c72eef3c => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:19.900396 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2e3c7-a172-4c97-b049-ed2c47a77100 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:19.902583 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d6a2e3c7-a172-4c97-b049-ed2c47a77100 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:19.935184 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2e3c7-a172-4c97-b049-ed2c47a77100 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:19.945813 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905cce58-0ffd-4c26-9bcf-1e4842ff12bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:19.945970 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:19.948214 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905cce58-0ffd-4c26-9bcf-1e4842ff12bf service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:19.949070 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905cce58-0ffd-4c26-9bcf-1e4842ff12bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:19.949552 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905cce58-0ffd-4c26-9bcf-1e4842ff12bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:19.953525 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-905cce58-0ffd-4c26-9bcf-1e4842ff12bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:19.997892 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb4beac-c413-4d94-b454-149a3aab73f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:19.997892 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb4beac-c413-4d94-b454-149a3aab73f1 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:20.009919 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 962/1921] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:07:20.011379 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea0c0ac-af92-49ef-8b90-a6d270b9f4ba None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:20.011740 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea0c0ac-af92-49ef-8b90-a6d270b9f4ba None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:20.012087 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea0c0ac-af92-49ef-8b90-a6d270b9f4ba None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:20.013942 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 963/1922] 10.222.0.27 () {54 vars in 921 bytes} [Sun Sep 4 01:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:07:20.015045 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378a6d3-5090-45fe-93cc-d9542053463c None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:20.015396 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378a6d3-5090-45fe-93cc-d9542053463c None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:20.016095 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378a6d3-5090-45fe-93cc-d9542053463c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:20.040360 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-0378a6d3-5090-45fe-93cc-d9542053463c None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:20.041673 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-0378a6d3-5090-45fe-93cc-d9542053463c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:20.052497 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 964/1923] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:20.053608 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912c17a-f427-408f-8d28-4209362f1717 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:20.054035 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912c17a-f427-408f-8d28-4209362f1717 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:20.054450 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912c17a-f427-408f-8d28-4209362f1717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:20.081020 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-6912c17a-f427-408f-8d28-4209362f1717 None None] MFA Rules not processed for user `68bc00bda5194cd8a7928f8c4df4ccd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:20.098097 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-6912c17a-f427-408f-8d28-4209362f1717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:20.098468 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905cce58-0ffd-4c26-9bcf-1e4842ff12bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:20.098468 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905cce58-0ffd-4c26-9bcf-1e4842ff12bf service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:20.107401 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 960/1924] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:07:20.108942 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a1a29-19ec-400e-8efd-e959c7a743b4 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:20.109339 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a1a29-19ec-400e-8efd-e959c7a743b4 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:20.109688 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a1a29-19ec-400e-8efd-e959c7a743b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:20.135768 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-be9a1a29-19ec-400e-8efd-e959c7a743b4 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:20.137273 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-be9a1a29-19ec-400e-8efd-e959c7a743b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:20.148311 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 961/1925] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:20.153766 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a1a29-19ec-400e-8efd-e959c7a743b4 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:20.172004 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a1a29-19ec-400e-8efd-e959c7a743b4 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:20.178978 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ea60e-32fc-43ee-bc09-8f52ce270800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:20.180151 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ea60e-32fc-43ee-bc09-8f52ce270800 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:20.180447 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ea60e-32fc-43ee-bc09-8f52ce270800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:20.180771 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ea60e-32fc-43ee-bc09-8f52ce270800 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:20.181433 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ea60e-32fc-43ee-bc09-8f52ce270800 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:07:20.181742 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ea60e-32fc-43ee-bc09-8f52ce270800 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:20.183909 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ea60e-32fc-43ee-bc09-8f52ce270800 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:20.202772 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 965/1926] 10.222.0.27 () {58 vars in 1018 bytes} [Sun Sep 4 01:07:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:20.205264 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 962/1927] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:07:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:07:20.207137 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6912c17a-f427-408f-8d28-4209362f1717 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:20.217991 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ea60e-32fc-43ee-bc09-8f52ce270800 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:20.226107 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6912c17a-f427-408f-8d28-4209362f1717 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:20.232084 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a83ea60e-32fc-43ee-bc09-8f52ce270800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:20.238026 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de279d34-4fe7-431c-82b5-a9c9125ff137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:20.242011 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de279d34-4fe7-431c-82b5-a9c9125ff137 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:20.242011 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de279d34-4fe7-431c-82b5-a9c9125ff137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:20.242011 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de279d34-4fe7-431c-82b5-a9c9125ff137 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:20.242011 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de279d34-4fe7-431c-82b5-a9c9125ff137 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:20.243195 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de279d34-4fe7-431c-82b5-a9c9125ff137 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:20.249254 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ea60e-32fc-43ee-bc09-8f52ce270800 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:20.255952 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9e1c5-e171-40a4-88c7-70ccc1cb4bb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:20.256438 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9e1c5-e171-40a4-88c7-70ccc1cb4bb0 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:20.256754 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9e1c5-e171-40a4-88c7-70ccc1cb4bb0 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:20.257328 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9e1c5-e171-40a4-88c7-70ccc1cb4bb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:20.266391 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad9e1c5-e171-40a4-88c7-70ccc1cb4bb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:20.270546 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad9e1c5-e171-40a4-88c7-70ccc1cb4bb0 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:20.280231 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 963/1928] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 01:07:20.372267 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 966/1929] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:07:20 2022] POST /identity/v3/projects => generated 387 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:20.373589 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9e1c5-e171-40a4-88c7-70ccc1cb4bb0 service nova] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:20.393843 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9e1c5-e171-40a4-88c7-70ccc1cb4bb0 service nova] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:20.461867 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218d837e-c690-4a41-a000-e46c26752f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:20.462512 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d837e-c690-4a41-a000-e46c26752f14 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:20.462792 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d837e-c690-4a41-a000-e46c26752f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:20.463065 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d837e-c690-4a41-a000-e46c26752f14 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:20.464102 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218d837e-c690-4a41-a000-e46c26752f14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:20.466078 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218d837e-c690-4a41-a000-e46c26752f14 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:20.533824 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de279d34-4fe7-431c-82b5-a9c9125ff137 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:20.533824 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-de279d34-4fe7-431c-82b5-a9c9125ff137 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:20.545406 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de279d34-4fe7-431c-82b5-a9c9125ff137 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:20.552609 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7d6ae2-4877-4bb4-ad3a-36615bb01b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:20.556105 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d6ae2-4877-4bb4-ad3a-36615bb01b92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:20.556105 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d6ae2-4877-4bb4-ad3a-36615bb01b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:20.556105 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d6ae2-4877-4bb4-ad3a-36615bb01b92 admin admin] PATH_INFO: `/v3/projects/baf79e5942dc4005a95062175e0178ea` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:20.556105 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7d6ae2-4877-4bb4-ad3a-36615bb01b92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=baf79e5942dc4005a95062175e0178ea)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:20.557531 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7d6ae2-4877-4bb4-ad3a-36615bb01b92 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:20.590313 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 964/1930] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:07:20 2022] POST /identity/v3/users => generated 294 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:07:20.594285 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218d837e-c690-4a41-a000-e46c26752f14 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:20.613712 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218d837e-c690-4a41-a000-e46c26752f14 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:20.619138 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab43edc5-c924-435a-a776-9ff9451d6a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:20.620293 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43edc5-c924-435a-a776-9ff9451d6a8b admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:20.620683 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43edc5-c924-435a-a776-9ff9451d6a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:20.621025 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43edc5-c924-435a-a776-9ff9451d6a8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:20.621672 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab43edc5-c924-435a-a776-9ff9451d6a8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:20.623272 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab43edc5-c924-435a-a776-9ff9451d6a8b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:20.636514 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 965/1931] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:20 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:20.636514 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab43edc5-c924-435a-a776-9ff9451d6a8b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:20.637510 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ab43edc5-c924-435a-a776-9ff9451d6a8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:20.650786 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab43edc5-c924-435a-a776-9ff9451d6a8b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:20.655454 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546462a7-5ef3-4a5e-9a46-58bd4284d139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:20.656077 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546462a7-5ef3-4a5e-9a46-58bd4284d139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:20.656412 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546462a7-5ef3-4a5e-9a46-58bd4284d139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:20.656651 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546462a7-5ef3-4a5e-9a46-58bd4284d139 admin admin] PATH_INFO: `/v3/projects/f2c413bb59fd4322a7c493ecbad719bd` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:20.658269 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546462a7-5ef3-4a5e-9a46-58bd4284d139 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2c413bb59fd4322a7c493ecbad719bd)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:20.660155 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546462a7-5ef3-4a5e-9a46-58bd4284d139 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:20.702503 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:20.711326 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-5a7d6ae2-4877-4bb4-ad3a-36615bb01b92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'baf79e5942dc4005a95062175e0178ea', 'request_id': 'req-5a7d6ae2-4877-4bb4-ad3a-36615bb01b92'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:20.794204 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-546462a7-5ef3-4a5e-9a46-58bd4284d139 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2c413bb59fd4322a7c493ecbad719bd', 'request_id': 'req-546462a7-5ef3-4a5e-9a46-58bd4284d139'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:20.802236 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-5a7d6ae2-4877-4bb4-ad3a-36615bb01b92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'baf79e5942dc4005a95062175e0178ea', 'request_id': 'req-5a7d6ae2-4877-4bb4-ad3a-36615bb01b92'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:20.808407 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-5a7d6ae2-4877-4bb4-ad3a-36615bb01b92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'baf79e5942dc4005a95062175e0178ea', 'request_id': 'req-5a7d6ae2-4877-4bb4-ad3a-36615bb01b92'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:20.820651 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-5a7d6ae2-4877-4bb4-ad3a-36615bb01b92 admin admin] The token cache is being invalidate because project baf79e5942dc4005a95062175e0178ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:07:20.820941 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-5a7d6ae2-4877-4bb4-ad3a-36615bb01b92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:20.821610 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-5a7d6ae2-4877-4bb4-ad3a-36615bb01b92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:20.823130 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 967/1932] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:07:20 2022] DELETE /identity/v3/projects/baf79e5942dc4005a95062175e0178ea => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:20.824660 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7d6ae2-4877-4bb4-ad3a-36615bb01b92 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:20.827276 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5a7d6ae2-4877-4bb4-ad3a-36615bb01b92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:20.896057 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7d6ae2-4877-4bb4-ad3a-36615bb01b92 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:20.922438 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-546462a7-5ef3-4a5e-9a46-58bd4284d139 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2c413bb59fd4322a7c493ecbad719bd', 'request_id': 'req-546462a7-5ef3-4a5e-9a46-58bd4284d139'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:20.929615 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-546462a7-5ef3-4a5e-9a46-58bd4284d139 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2c413bb59fd4322a7c493ecbad719bd', 'request_id': 'req-546462a7-5ef3-4a5e-9a46-58bd4284d139'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:20.942961 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-546462a7-5ef3-4a5e-9a46-58bd4284d139 admin admin] The token cache is being invalidate because project f2c413bb59fd4322a7c493ecbad719bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:07:20.943254 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-546462a7-5ef3-4a5e-9a46-58bd4284d139 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:20.944014 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-546462a7-5ef3-4a5e-9a46-58bd4284d139 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:20.945629 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 966/1933] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:07:20 2022] DELETE /identity/v3/projects/f2c413bb59fd4322a7c493ecbad719bd => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:20.946464 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546462a7-5ef3-4a5e-9a46-58bd4284d139 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:20.947796 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-546462a7-5ef3-4a5e-9a46-58bd4284d139 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:20.966734 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81566f-865c-4ba8-85ae-b75ceccdc69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:20.967370 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81566f-865c-4ba8-85ae-b75ceccdc69d admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:20.968202 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81566f-865c-4ba8-85ae-b75ceccdc69d admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:20.968544 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81566f-865c-4ba8-85ae-b75ceccdc69d admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:20.969153 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81566f-865c-4ba8-85ae-b75ceccdc69d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:20.974067 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81566f-865c-4ba8-85ae-b75ceccdc69d admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:20.990798 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 968/1934] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:20 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:20.996130 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81566f-865c-4ba8-85ae-b75ceccdc69d admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:20.997922 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ff81566f-865c-4ba8-85ae-b75ceccdc69d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:21.033325 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546462a7-5ef3-4a5e-9a46-58bd4284d139 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:21.033497 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81566f-865c-4ba8-85ae-b75ceccdc69d admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:21.038260 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d0990-0a01-47cc-91b0-96a06326bfad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:21.039024 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d0990-0a01-47cc-91b0-96a06326bfad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:21.039370 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d0990-0a01-47cc-91b0-96a06326bfad admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:21.039735 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d0990-0a01-47cc-91b0-96a06326bfad admin admin] PATH_INFO: `/v3/users/f64a51e769884b5dba887808070d2848` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:21.041169 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ff7fe-d5ff-4b27-a764-1e3d7a9a05a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:21.041283 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d0990-0a01-47cc-91b0-96a06326bfad admin admin] RBAC: Authorizing `identity:delete_user(user_id=f64a51e769884b5dba887808070d2848)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:21.042105 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ff7fe-d5ff-4b27-a764-1e3d7a9a05a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:21.042532 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ff7fe-d5ff-4b27-a764-1e3d7a9a05a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:21.042816 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ff7fe-d5ff-4b27-a764-1e3d7a9a05a4 admin admin] PATH_INFO: `/v3/users/74e72860d6a64b7ab521a297ef8359d0` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:21.043560 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d0990-0a01-47cc-91b0-96a06326bfad admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:21.044082 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ff7fe-d5ff-4b27-a764-1e3d7a9a05a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74e72860d6a64b7ab521a297ef8359d0)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:21.047134 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ff7fe-d5ff-4b27-a764-1e3d7a9a05a4 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:21.239839 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-108d0990-0a01-47cc-91b0-96a06326bfad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f64a51e769884b5dba887808070d2848'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:21.240613 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-108d0990-0a01-47cc-91b0-96a06326bfad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f64a51e769884b5dba887808070d2848'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:21.265369 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e10ff7fe-d5ff-4b27-a764-1e3d7a9a05a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74e72860d6a64b7ab521a297ef8359d0'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:21.266024 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e10ff7fe-d5ff-4b27-a764-1e3d7a9a05a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74e72860d6a64b7ab521a297ef8359d0'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:21.273595 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e10ff7fe-d5ff-4b27-a764-1e3d7a9a05a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74e72860d6a64b7ab521a297ef8359d0'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:21.302072 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-108d0990-0a01-47cc-91b0-96a06326bfad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f64a51e769884b5dba887808070d2848'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:21.302658 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-108d0990-0a01-47cc-91b0-96a06326bfad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f64a51e769884b5dba887808070d2848'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:21.316114 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-108d0990-0a01-47cc-91b0-96a06326bfad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f64a51e769884b5dba887808070d2848'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:21.326331 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:21.328659 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 969/1935] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:07:20 2022] DELETE /identity/v3/users/f64a51e769884b5dba887808070d2848 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:21.332923 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d0990-0a01-47cc-91b0-96a06326bfad admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:21.334284 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-108d0990-0a01-47cc-91b0-96a06326bfad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:21.348330 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e10ff7fe-d5ff-4b27-a764-1e3d7a9a05a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74e72860d6a64b7ab521a297ef8359d0'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:21.349181 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e10ff7fe-d5ff-4b27-a764-1e3d7a9a05a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74e72860d6a64b7ab521a297ef8359d0'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:21.379234 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 967/1936] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:07:20 2022] DELETE /identity/v3/users/74e72860d6a64b7ab521a297ef8359d0 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:21.405668 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d0990-0a01-47cc-91b0-96a06326bfad admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:21.470289 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5efcdd-cdf2-476e-8fb4-f4e8755442a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:21.471074 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5efcdd-cdf2-476e-8fb4-f4e8755442a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:21.471372 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5efcdd-cdf2-476e-8fb4-f4e8755442a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:21.471772 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5efcdd-cdf2-476e-8fb4-f4e8755442a0 admin admin] PATH_INFO: `/v3/projects/44bb8d643c484e3683a83249575277cb/users/4110e5346972427fb5a0a4cfae90ad3e/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:21.490177 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5efcdd-cdf2-476e-8fb4-f4e8755442a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44bb8d643c484e3683a83249575277cb, user_id=4110e5346972427fb5a0a4cfae90ad3e, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:21.493019 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5efcdd-cdf2-476e-8fb4-f4e8755442a0 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:21.596597 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 970/1937] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:21 2022] PUT /identity/v3/projects/44bb8d643c484e3683a83249575277cb/users/4110e5346972427fb5a0a4cfae90ad3e/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:21.603120 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e8873-65ba-477e-b0a7-4e681495f7f2 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:21.603640 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e8873-65ba-477e-b0a7-4e681495f7f2 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:21.604124 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e8873-65ba-477e-b0a7-4e681495f7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:21.650224 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-208e8873-65ba-477e-b0a7-4e681495f7f2 None None] MFA Rules not processed for user `4110e5346972427fb5a0a4cfae90ad3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:21.651775 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-208e8873-65ba-477e-b0a7-4e681495f7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:21.739359 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5efcdd-cdf2-476e-8fb4-f4e8755442a0 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:21.740497 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 968/1938] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:21.741517 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ee5efcdd-cdf2-476e-8fb4-f4e8755442a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:21.822810 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5efcdd-cdf2-476e-8fb4-f4e8755442a0 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:21.826081 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220c88c-372c-45d6-af4d-d9a5b78f43b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:21.826714 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220c88c-372c-45d6-af4d-d9a5b78f43b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:21.827066 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220c88c-372c-45d6-af4d-d9a5b78f43b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:21.827409 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220c88c-372c-45d6-af4d-d9a5b78f43b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:21.828146 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6220c88c-372c-45d6-af4d-d9a5b78f43b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:21.830975 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6220c88c-372c-45d6-af4d-d9a5b78f43b1 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:21.911340 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e8873-65ba-477e-b0a7-4e681495f7f2 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:21.913566 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-208e8873-65ba-477e-b0a7-4e681495f7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:21.930552 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e8873-65ba-477e-b0a7-4e681495f7f2 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:21.942547 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a495b73-441e-4c9a-90b7-6bba58451ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:21.942547 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a495b73-441e-4c9a-90b7-6bba58451ed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:21.942547 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a495b73-441e-4c9a-90b7-6bba58451ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:21.942547 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a495b73-441e-4c9a-90b7-6bba58451ed2 admin admin] PATH_INFO: `/v3/projects/e5adef07772a42e9a4fdb7800aaee4c9` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:21.942967 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 971/1939] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:07:21 2022] POST /identity/v3/projects => generated 387 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:21.942967 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220c88c-372c-45d6-af4d-d9a5b78f43b1 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:21.943341 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-6220c88c-372c-45d6-af4d-d9a5b78f43b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:21.944391 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a495b73-441e-4c9a-90b7-6bba58451ed2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5adef07772a42e9a4fdb7800aaee4c9)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:21.946551 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a495b73-441e-4c9a-90b7-6bba58451ed2 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:22.034537 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220c88c-372c-45d6-af4d-d9a5b78f43b1 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:22.041428 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca01f9ab-fec7-42b2-9dd0-3d6e1cb196de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:22.042182 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca01f9ab-fec7-42b2-9dd0-3d6e1cb196de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:22.042362 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca01f9ab-fec7-42b2-9dd0-3d6e1cb196de admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:22.042681 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca01f9ab-fec7-42b2-9dd0-3d6e1cb196de admin admin] PATH_INFO: `/v3/projects/6296c1ca3139499f85797dc5f7120cf0` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:22.043850 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca01f9ab-fec7-42b2-9dd0-3d6e1cb196de admin admin] RBAC: Authorizing `identity:delete_project(project_id=6296c1ca3139499f85797dc5f7120cf0)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:22.045780 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca01f9ab-fec7-42b2-9dd0-3d6e1cb196de admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:22.130091 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2a495b73-441e-4c9a-90b7-6bba58451ed2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5adef07772a42e9a4fdb7800aaee4c9', 'request_id': 'req-2a495b73-441e-4c9a-90b7-6bba58451ed2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:22.165515 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ca01f9ab-fec7-42b2-9dd0-3d6e1cb196de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6296c1ca3139499f85797dc5f7120cf0', 'request_id': 'req-ca01f9ab-fec7-42b2-9dd0-3d6e1cb196de'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:22.174044 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:22.231083 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2a495b73-441e-4c9a-90b7-6bba58451ed2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5adef07772a42e9a4fdb7800aaee4c9', 'request_id': 'req-2a495b73-441e-4c9a-90b7-6bba58451ed2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:22.239422 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2a495b73-441e-4c9a-90b7-6bba58451ed2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5adef07772a42e9a4fdb7800aaee4c9', 'request_id': 'req-2a495b73-441e-4c9a-90b7-6bba58451ed2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:22.251999 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2a495b73-441e-4c9a-90b7-6bba58451ed2 admin admin] The token cache is being invalidate because project e5adef07772a42e9a4fdb7800aaee4c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:07:22.252655 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2a495b73-441e-4c9a-90b7-6bba58451ed2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:22.253477 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2a495b73-441e-4c9a-90b7-6bba58451ed2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:22.255130 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 969/1940] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:07:21 2022] DELETE /identity/v3/projects/e5adef07772a42e9a4fdb7800aaee4c9 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:22.256322 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a495b73-441e-4c9a-90b7-6bba58451ed2 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:22.257733 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2a495b73-441e-4c9a-90b7-6bba58451ed2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:22.342828 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a495b73-441e-4c9a-90b7-6bba58451ed2 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:22.347557 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ca01f9ab-fec7-42b2-9dd0-3d6e1cb196de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6296c1ca3139499f85797dc5f7120cf0', 'request_id': 'req-ca01f9ab-fec7-42b2-9dd0-3d6e1cb196de'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:22.354919 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ca01f9ab-fec7-42b2-9dd0-3d6e1cb196de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6296c1ca3139499f85797dc5f7120cf0', 'request_id': 'req-ca01f9ab-fec7-42b2-9dd0-3d6e1cb196de'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:22.369983 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ca01f9ab-fec7-42b2-9dd0-3d6e1cb196de admin admin] The token cache is being invalidate because project 6296c1ca3139499f85797dc5f7120cf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:07:22.370551 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ca01f9ab-fec7-42b2-9dd0-3d6e1cb196de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:22.371536 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ca01f9ab-fec7-42b2-9dd0-3d6e1cb196de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:22.377160 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 972/1941] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:07:21 2022] DELETE /identity/v3/projects/6296c1ca3139499f85797dc5f7120cf0 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:22.377160 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca01f9ab-fec7-42b2-9dd0-3d6e1cb196de admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:22.392448 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ca01f9ab-fec7-42b2-9dd0-3d6e1cb196de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:22.436103 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff958e6-574e-43c0-b168-d99af41cf436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:22.437026 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff958e6-574e-43c0-b168-d99af41cf436 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:22.437406 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff958e6-574e-43c0-b168-d99af41cf436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:22.437774 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff958e6-574e-43c0-b168-d99af41cf436 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:22.438510 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff958e6-574e-43c0-b168-d99af41cf436 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:22.441882 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff958e6-574e-43c0-b168-d99af41cf436 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:22.494689 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca01f9ab-fec7-42b2-9dd0-3d6e1cb196de admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:22.499331 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c55b1-868a-4c06-8abf-14873be0651f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:22.499961 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c55b1-868a-4c06-8abf-14873be0651f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:22.500242 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c55b1-868a-4c06-8abf-14873be0651f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:22.500549 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c55b1-868a-4c06-8abf-14873be0651f admin admin] PATH_INFO: `/v3/users/d76ec7c3115745c9824df8b1823a87c4` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:22.502696 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0c55b1-868a-4c06-8abf-14873be0651f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d76ec7c3115745c9824df8b1823a87c4)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:22.504738 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0c55b1-868a-4c06-8abf-14873be0651f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:22.519808 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 970/1942] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:07:22 2022] POST /identity/v3/users => generated 294 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:22.522181 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4521c3b-fe5b-46c4-ab70-9cfd735b57eb None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:22.522181 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4521c3b-fe5b-46c4-ab70-9cfd735b57eb None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:22.522181 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4521c3b-fe5b-46c4-ab70-9cfd735b57eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:22.540892 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-d4521c3b-fe5b-46c4-ab70-9cfd735b57eb None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:22.545199 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d4521c3b-fe5b-46c4-ab70-9cfd735b57eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:22.550595 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 971/1943] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:22.551780 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede94282-2a07-4737-9bcd-b1a6971f3f41 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:22.552060 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede94282-2a07-4737-9bcd-b1a6971f3f41 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:22.552324 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede94282-2a07-4737-9bcd-b1a6971f3f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:22.571465 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-ede94282-2a07-4737-9bcd-b1a6971f3f41 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:22.573110 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ede94282-2a07-4737-9bcd-b1a6971f3f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:22.581110 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 972/1944] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:22.581902 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede94282-2a07-4737-9bcd-b1a6971f3f41 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:22.583107 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ede94282-2a07-4737-9bcd-b1a6971f3f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:22.594529 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede94282-2a07-4737-9bcd-b1a6971f3f41 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:22.599461 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982f630-71d1-4285-86a5-67b7209e2986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:22.600089 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982f630-71d1-4285-86a5-67b7209e2986 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:22.600369 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982f630-71d1-4285-86a5-67b7209e2986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:22.600675 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982f630-71d1-4285-86a5-67b7209e2986 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:22.601327 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7982f630-71d1-4285-86a5-67b7209e2986 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:22.602890 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7982f630-71d1-4285-86a5-67b7209e2986 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:22.614014 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 973/1945] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:22.614426 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1ce2f-e3f3-4d43-9d7e-3a9594814160 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:22.614707 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1ce2f-e3f3-4d43-9d7e-3a9594814160 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:22.614997 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1ce2f-e3f3-4d43-9d7e-3a9594814160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:22.634022 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-d1c1ce2f-e3f3-4d43-9d7e-3a9594814160 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:22.635229 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d1c1ce2f-e3f3-4d43-9d7e-3a9594814160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:22.643399 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 974/1946] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:22.644830 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d0498-5c49-4b51-923d-fe83daaf08e9 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:22.644830 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d0498-5c49-4b51-923d-fe83daaf08e9 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:22.647682 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d0498-5c49-4b51-923d-fe83daaf08e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:22.665543 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c56d0498-5c49-4b51-923d-fe83daaf08e9 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:22.666756 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c56d0498-5c49-4b51-923d-fe83daaf08e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:22.675376 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 975/1947] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:22.676213 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d0498-5c49-4b51-923d-fe83daaf08e9 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:22.685804 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d0498-5c49-4b51-923d-fe83daaf08e9 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:22.690520 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0b05d-93c7-41b7-bb26-e19b55a72ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:22.694456 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0b05d-93c7-41b7-bb26-e19b55a72ec2 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:22.694456 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0b05d-93c7-41b7-bb26-e19b55a72ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:22.694456 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0b05d-93c7-41b7-bb26-e19b55a72ec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:22.694456 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce0b05d-93c7-41b7-bb26-e19b55a72ec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:22.694456 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce0b05d-93c7-41b7-bb26-e19b55a72ec2 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:22.704799 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 976/1948] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:22.705443 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0b05d-93c7-41b7-bb26-e19b55a72ec2 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:22.716472 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0b05d-93c7-41b7-bb26-e19b55a72ec2 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:22.722655 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2852e4-9d6f-4666-a228-e9f60fcbaf28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:22.723557 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2852e4-9d6f-4666-a228-e9f60fcbaf28 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:22.723992 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2852e4-9d6f-4666-a228-e9f60fcbaf28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:22.724319 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2852e4-9d6f-4666-a228-e9f60fcbaf28 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:22.724986 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2852e4-9d6f-4666-a228-e9f60fcbaf28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:07:22.725290 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2852e4-9d6f-4666-a228-e9f60fcbaf28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:22.726856 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2852e4-9d6f-4666-a228-e9f60fcbaf28 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:22.728277 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0f0c55b1-868a-4c06-8abf-14873be0651f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd76ec7c3115745c9824df8b1823a87c4'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:22.729160 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0f0c55b1-868a-4c06-8abf-14873be0651f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd76ec7c3115745c9824df8b1823a87c4'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:22.739616 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:22.743233 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 977/1949] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:07:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:07:22.743994 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2852e4-9d6f-4666-a228-e9f60fcbaf28 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:22.745638 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-aa2852e4-9d6f-4666-a228-e9f60fcbaf28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:22.758178 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2852e4-9d6f-4666-a228-e9f60fcbaf28 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:22.762698 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dfd596-2542-4a18-9fc0-44a2e5d625e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:22.763264 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfd596-2542-4a18-9fc0-44a2e5d625e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:22.763793 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfd596-2542-4a18-9fc0-44a2e5d625e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:22.764125 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfd596-2542-4a18-9fc0-44a2e5d625e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:22.764713 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dfd596-2542-4a18-9fc0-44a2e5d625e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:07:22.768028 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dfd596-2542-4a18-9fc0-44a2e5d625e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:22.768028 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dfd596-2542-4a18-9fc0-44a2e5d625e0 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:22.777781 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 978/1950] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:07:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:07:22.778525 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dfd596-2542-4a18-9fc0-44a2e5d625e0 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:22.780133 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b8dfd596-2542-4a18-9fc0-44a2e5d625e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:22.787397 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0f0c55b1-868a-4c06-8abf-14873be0651f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd76ec7c3115745c9824df8b1823a87c4'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:22.788109 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0f0c55b1-868a-4c06-8abf-14873be0651f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd76ec7c3115745c9824df8b1823a87c4'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:22.793460 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dfd596-2542-4a18-9fc0-44a2e5d625e0 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:22.799111 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e063d-7d2d-47da-891f-31e6a4490b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:22.799775 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e063d-7d2d-47da-891f-31e6a4490b08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:22.800059 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e063d-7d2d-47da-891f-31e6a4490b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:22.800329 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e063d-7d2d-47da-891f-31e6a4490b08 admin admin] PATH_INFO: `/v3/projects/9799f42352f349bb8037b270cf3b48db/users/5a7cfec29d1442b4b6871eff57480187/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:22.802664 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0f0c55b1-868a-4c06-8abf-14873be0651f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd76ec7c3115745c9824df8b1823a87c4'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:22.814852 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e063d-7d2d-47da-891f-31e6a4490b08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9799f42352f349bb8037b270cf3b48db, user_id=5a7cfec29d1442b4b6871eff57480187, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:22.815369 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 973/1951] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:07:22 2022] DELETE /identity/v3/users/d76ec7c3115745c9824df8b1823a87c4 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:22.816119 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c55b1-868a-4c06-8abf-14873be0651f admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:22.817499 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-0f0c55b1-868a-4c06-8abf-14873be0651f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:22.817833 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e063d-7d2d-47da-891f-31e6a4490b08 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:22.879784 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c55b1-868a-4c06-8abf-14873be0651f admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:22.884225 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e5f5d-bb0a-4d59-9732-c246667e905f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:22.884871 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e5f5d-bb0a-4d59-9732-c246667e905f admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:22.885148 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e5f5d-bb0a-4d59-9732-c246667e905f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:22.885418 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e5f5d-bb0a-4d59-9732-c246667e905f admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:22.886038 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618e5f5d-bb0a-4d59-9732-c246667e905f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:22.887916 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618e5f5d-bb0a-4d59-9732-c246667e905f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:22.908696 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 979/1952] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:22 2022] PUT /identity/v3/projects/9799f42352f349bb8037b270cf3b48db/users/5a7cfec29d1442b4b6871eff57480187/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:22.909691 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e063d-7d2d-47da-891f-31e6a4490b08 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:22.974150 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e063d-7d2d-47da-891f-31e6a4490b08 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:22.978878 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace88127-c816-443f-8c2d-cf175305248c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:22.979483 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace88127-c816-443f-8c2d-cf175305248c admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:22.979764 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace88127-c816-443f-8c2d-cf175305248c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:22.980092 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace88127-c816-443f-8c2d-cf175305248c admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:22.980750 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace88127-c816-443f-8c2d-cf175305248c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:22.982676 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace88127-c816-443f-8c2d-cf175305248c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:22.994891 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 974/1953] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:07:22 2022] POST /identity/v3/projects => generated 363 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:22.996245 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9155545-7e9d-452a-9f6e-1f1c04bb1477 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:22.996565 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9155545-7e9d-452a-9f6e-1f1c04bb1477 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:22.996844 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9155545-7e9d-452a-9f6e-1f1c04bb1477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:23.016730 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-c9155545-7e9d-452a-9f6e-1f1c04bb1477 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:23.018635 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c9155545-7e9d-452a-9f6e-1f1c04bb1477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:23.076257 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 975/1954] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:23.078665 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81fe6a-101f-422d-a80f-35b122c9e9ac None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:23.078665 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81fe6a-101f-422d-a80f-35b122c9e9ac None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:23.078665 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81fe6a-101f-422d-a80f-35b122c9e9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:23.088971 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 980/1955] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:07:22 2022] POST /identity/v3/projects => generated 379 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:23.089856 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace88127-c816-443f-8c2d-cf175305248c admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:23.109254 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-3f81fe6a-101f-422d-a80f-35b122c9e9ac None None] MFA Rules not processed for user `5a7cfec29d1442b4b6871eff57480187`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:23.110506 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3f81fe6a-101f-422d-a80f-35b122c9e9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:23.179292 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace88127-c816-443f-8c2d-cf175305248c admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:23.181843 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 976/1956] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:23.185392 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81fe6a-101f-422d-a80f-35b122c9e9ac None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:23.185392 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3f81fe6a-101f-422d-a80f-35b122c9e9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:23.185560 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241611d5-194e-4db1-b0d3-9f4b347c43f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:23.185560 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241611d5-194e-4db1-b0d3-9f4b347c43f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:23.185560 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241611d5-194e-4db1-b0d3-9f4b347c43f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:23.185782 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241611d5-194e-4db1-b0d3-9f4b347c43f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:23.186361 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241611d5-194e-4db1-b0d3-9f4b347c43f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:23.188286 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241611d5-194e-4db1-b0d3-9f4b347c43f8 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:23.244506 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81fe6a-101f-422d-a80f-35b122c9e9ac None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:23.248292 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49fc8b5-2edb-42d5-b089-4fa49fed68b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:23.248995 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49fc8b5-2edb-42d5-b089-4fa49fed68b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:23.249279 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49fc8b5-2edb-42d5-b089-4fa49fed68b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:23.249558 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49fc8b5-2edb-42d5-b089-4fa49fed68b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:23.260088 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49fc8b5-2edb-42d5-b089-4fa49fed68b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:23.261740 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49fc8b5-2edb-42d5-b089-4fa49fed68b4 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:23.273772 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 977/1957] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:07:23.274577 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49fc8b5-2edb-42d5-b089-4fa49fed68b4 service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:23.284748 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49fc8b5-2edb-42d5-b089-4fa49fed68b4 service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:23.289668 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ec497-fab9-4fbd-a1ef-6081659b431e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:23.290268 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ec497-fab9-4fbd-a1ef-6081659b431e admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:23.290550 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ec497-fab9-4fbd-a1ef-6081659b431e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:23.290827 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ec497-fab9-4fbd-a1ef-6081659b431e admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:23.291452 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412ec497-fab9-4fbd-a1ef-6081659b431e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:23.293411 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412ec497-fab9-4fbd-a1ef-6081659b431e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:23.309997 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 981/1958] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:07:23 2022] POST /identity/v3/users => generated 282 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:23.312592 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241611d5-194e-4db1-b0d3-9f4b347c43f8 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:23.323662 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241611d5-194e-4db1-b0d3-9f4b347c43f8 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:23.330182 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907869a-4dd0-493e-9b1c-0d31a5b8ad29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:23.330873 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907869a-4dd0-493e-9b1c-0d31a5b8ad29 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:23.331226 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907869a-4dd0-493e-9b1c-0d31a5b8ad29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:23.331849 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907869a-4dd0-493e-9b1c-0d31a5b8ad29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:23.332498 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4907869a-4dd0-493e-9b1c-0d31a5b8ad29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:23.334401 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4907869a-4dd0-493e-9b1c-0d31a5b8ad29 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:23.405497 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 978/1959] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:07:23 2022] POST /identity/v3/users => generated 290 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:23.407202 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ec497-fab9-4fbd-a1ef-6081659b431e admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:23.427081 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 982/1960] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:07:23 2022] POST /identity/v3/projects => generated 387 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:23.428874 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907869a-4dd0-493e-9b1c-0d31a5b8ad29 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:23.439524 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ec497-fab9-4fbd-a1ef-6081659b431e admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:23.518021 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fccef9d-6ef1-4887-a5e6-ffb2f178f7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:23.518021 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fccef9d-6ef1-4887-a5e6-ffb2f178f7b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:23.518021 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fccef9d-6ef1-4887-a5e6-ffb2f178f7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:23.518788 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907869a-4dd0-493e-9b1c-0d31a5b8ad29 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:23.518866 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fccef9d-6ef1-4887-a5e6-ffb2f178f7b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:23.518866 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fccef9d-6ef1-4887-a5e6-ffb2f178f7b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:23.519661 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fccef9d-6ef1-4887-a5e6-ffb2f178f7b9 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:23.523165 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3d2fa-da2c-48ca-a019-6ada62b149da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:23.523869 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3d2fa-da2c-48ca-a019-6ada62b149da admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:23.524223 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3d2fa-da2c-48ca-a019-6ada62b149da admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:23.524623 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3d2fa-da2c-48ca-a019-6ada62b149da admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:23.525669 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b3d2fa-da2c-48ca-a019-6ada62b149da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:23.527610 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b3d2fa-da2c-48ca-a019-6ada62b149da admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:23.536481 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 979/1961] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:23 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:23.536481 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fccef9d-6ef1-4887-a5e6-ffb2f178f7b9 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:23.542974 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 983/1962] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:23 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:23.544100 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3d2fa-da2c-48ca-a019-6ada62b149da admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:23.550704 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fccef9d-6ef1-4887-a5e6-ffb2f178f7b9 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:23.556607 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3d2fa-da2c-48ca-a019-6ada62b149da admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:23.557085 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ebf2c0-1235-4386-94bc-f59b5fa638e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:23.558043 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ebf2c0-1235-4386-94bc-f59b5fa638e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:23.558331 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ebf2c0-1235-4386-94bc-f59b5fa638e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:23.558691 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ebf2c0-1235-4386-94bc-f59b5fa638e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:23.559461 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ebf2c0-1235-4386-94bc-f59b5fa638e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:23.561818 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ebf2c0-1235-4386-94bc-f59b5fa638e2 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:23.562611 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13841067-ed42-49dc-8276-13117746f704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:23.564652 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13841067-ed42-49dc-8276-13117746f704 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:23.564652 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13841067-ed42-49dc-8276-13117746f704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:23.564830 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13841067-ed42-49dc-8276-13117746f704 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:23.567401 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13841067-ed42-49dc-8276-13117746f704 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:23.571740 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13841067-ed42-49dc-8276-13117746f704 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:23.584329 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 984/1963] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:23 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:23.584997 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13841067-ed42-49dc-8276-13117746f704 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:23.597708 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13841067-ed42-49dc-8276-13117746f704 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:23.602470 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3402d67-ea4a-4986-b855-7c0ec3220052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:23.604595 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3402d67-ea4a-4986-b855-7c0ec3220052 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:23.604595 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3402d67-ea4a-4986-b855-7c0ec3220052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:23.604595 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3402d67-ea4a-4986-b855-7c0ec3220052 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:23.605215 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3402d67-ea4a-4986-b855-7c0ec3220052 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:23.606894 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3402d67-ea4a-4986-b855-7c0ec3220052 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:23.619579 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 985/1964] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:23.620293 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3402d67-ea4a-4986-b855-7c0ec3220052 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:23.634357 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3402d67-ea4a-4986-b855-7c0ec3220052 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:23.640508 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ad7ab-4b3f-4ef6-9ed3-7fafa2da2615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:23.641236 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ad7ab-4b3f-4ef6-9ed3-7fafa2da2615 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:23.641516 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ad7ab-4b3f-4ef6-9ed3-7fafa2da2615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:23.641919 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ad7ab-4b3f-4ef6-9ed3-7fafa2da2615 admin admin] PATH_INFO: `/v3/projects/fb85b90af9f44e3795934f331f7b1bcd/users/926f67677d0741f28736021f0ed52a85/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:23.668995 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686ad7ab-4b3f-4ef6-9ed3-7fafa2da2615 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb85b90af9f44e3795934f331f7b1bcd, user_id=926f67677d0741f28736021f0ed52a85, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:23.672392 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686ad7ab-4b3f-4ef6-9ed3-7fafa2da2615 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:23.711048 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 980/1965] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:07:23 2022] POST /identity/v3/users => generated 300 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:23.712757 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ebf2c0-1235-4386-94bc-f59b5fa638e2 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:23.725520 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ebf2c0-1235-4386-94bc-f59b5fa638e2 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:23.730779 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69de338-3e28-4c89-b3ba-80ded208cf99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:23.731833 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69de338-3e28-4c89-b3ba-80ded208cf99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:23.731833 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69de338-3e28-4c89-b3ba-80ded208cf99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:23.732231 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69de338-3e28-4c89-b3ba-80ded208cf99 admin admin] PATH_INFO: `/v3/projects/c30877420157436cb3c46ba6c9a61ad9/users/aebea04dcc6044978b6915dbcabcb01a/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:23.752092 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69de338-3e28-4c89-b3ba-80ded208cf99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c30877420157436cb3c46ba6c9a61ad9, user_id=aebea04dcc6044978b6915dbcabcb01a, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:23.756877 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69de338-3e28-4c89-b3ba-80ded208cf99 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:23.762409 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 986/1966] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:23 2022] PUT /identity/v3/projects/fb85b90af9f44e3795934f331f7b1bcd/users/926f67677d0741f28736021f0ed52a85/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:23.763836 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ad7ab-4b3f-4ef6-9ed3-7fafa2da2615 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:23.778232 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:23.817391 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:23.828333 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 981/1967] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:23 2022] PUT /identity/v3/projects/c30877420157436cb3c46ba6c9a61ad9/users/aebea04dcc6044978b6915dbcabcb01a/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:23.831333 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd2c0c-a3ac-4662-99ca-3d7985dd2915 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:23.832273 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd2c0c-a3ac-4662-99ca-3d7985dd2915 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:23.832661 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd2c0c-a3ac-4662-99ca-3d7985dd2915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:23.868286 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ad7ab-4b3f-4ef6-9ed3-7fafa2da2615 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:23.870860 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-e2dd2c0c-a3ac-4662-99ca-3d7985dd2915 None None] MFA Rules not processed for user `926f67677d0741f28736021f0ed52a85`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:23.872604 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e2dd2c0c-a3ac-4662-99ca-3d7985dd2915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:23.931567 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3357d719-f9a1-4bd0-bf07-55c450fc2e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:23.932290 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357d719-f9a1-4bd0-bf07-55c450fc2e4c admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:23.932670 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357d719-f9a1-4bd0-bf07-55c450fc2e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:23.933013 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357d719-f9a1-4bd0-bf07-55c450fc2e4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:23.933932 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3357d719-f9a1-4bd0-bf07-55c450fc2e4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:23.938567 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3357d719-f9a1-4bd0-bf07-55c450fc2e4c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:23.954181 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 982/1968] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:23.954181 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847ef8d-4974-457c-a121-3738b77b9c28 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:23.954181 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847ef8d-4974-457c-a121-3738b77b9c28 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:23.954432 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847ef8d-4974-457c-a121-3738b77b9c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:23.959272 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 987/1969] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:23 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:23.960140 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3357d719-f9a1-4bd0-bf07-55c450fc2e4c admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:23.964633 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3357d719-f9a1-4bd0-bf07-55c450fc2e4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:23.993468 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-7847ef8d-4974-457c-a121-3738b77b9c28 None None] MFA Rules not processed for user `aebea04dcc6044978b6915dbcabcb01a`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:23.993468 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-7847ef8d-4974-457c-a121-3738b77b9c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:23.995447 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3357d719-f9a1-4bd0-bf07-55c450fc2e4c admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:24.000574 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d9aadb-5bce-482b-871d-155083eeaf2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:24.002337 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d9aadb-5bce-482b-871d-155083eeaf2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:24.002822 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d9aadb-5bce-482b-871d-155083eeaf2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:24.003228 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d9aadb-5bce-482b-871d-155083eeaf2b admin admin] PATH_INFO: `/v3/projects/bc3ba581f882400dbc65d9a28d0de9fc` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:24.004895 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d9aadb-5bce-482b-871d-155083eeaf2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc3ba581f882400dbc65d9a28d0de9fc)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:24.006975 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d9aadb-5bce-482b-871d-155083eeaf2b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:24.064826 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 983/1970] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:24.067533 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847ef8d-4974-457c-a121-3738b77b9c28 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:24.080238 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847ef8d-4974-457c-a121-3738b77b9c28 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:24.085182 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4733eb1-da08-46c2-acc7-1329890699d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:24.085805 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4733eb1-da08-46c2-acc7-1329890699d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:24.086083 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4733eb1-da08-46c2-acc7-1329890699d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:24.086359 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4733eb1-da08-46c2-acc7-1329890699d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:24.086996 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4733eb1-da08-46c2-acc7-1329890699d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:24.089149 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4733eb1-da08-46c2-acc7-1329890699d8 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:24.200719 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-54d9aadb-5bce-482b-871d-155083eeaf2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc3ba581f882400dbc65d9a28d0de9fc', 'request_id': 'req-54d9aadb-5bce-482b-871d-155083eeaf2b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:24.247430 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 984/1971] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:07:24 2022] POST /identity/v3/projects => generated 365 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:24.248777 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4733eb1-da08-46c2-acc7-1329890699d8 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:24.302231 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-54d9aadb-5bce-482b-871d-155083eeaf2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc3ba581f882400dbc65d9a28d0de9fc', 'request_id': 'req-54d9aadb-5bce-482b-871d-155083eeaf2b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:24.308854 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-54d9aadb-5bce-482b-871d-155083eeaf2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc3ba581f882400dbc65d9a28d0de9fc', 'request_id': 'req-54d9aadb-5bce-482b-871d-155083eeaf2b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:24.326483 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4733eb1-da08-46c2-acc7-1329890699d8 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:24.332586 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-54d9aadb-5bce-482b-871d-155083eeaf2b admin admin] The token cache is being invalidate because project bc3ba581f882400dbc65d9a28d0de9fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:07:24.334297 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-54d9aadb-5bce-482b-871d-155083eeaf2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:24.334692 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc52a6-63f1-4c9b-a722-f72e916660e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:24.335445 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-54d9aadb-5bce-482b-871d-155083eeaf2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:24.335873 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc52a6-63f1-4c9b-a722-f72e916660e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:24.336154 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc52a6-63f1-4c9b-a722-f72e916660e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:24.336427 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc52a6-63f1-4c9b-a722-f72e916660e7 admin admin] PATH_INFO: `/v3/projects/39b32a56c3694cc1b1f5027ae1b123d2/users/9476793c690d45019109159350a3b850/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:24.340888 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 988/1972] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:07:23 2022] DELETE /identity/v3/projects/bc3ba581f882400dbc65d9a28d0de9fc => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:24.340888 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d9aadb-5bce-482b-871d-155083eeaf2b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:24.345540 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-54d9aadb-5bce-482b-871d-155083eeaf2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:24.363450 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cc52a6-63f1-4c9b-a722-f72e916660e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39b32a56c3694cc1b1f5027ae1b123d2, user_id=9476793c690d45019109159350a3b850, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:24.371167 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cc52a6-63f1-4c9b-a722-f72e916660e7 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:24.450429 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d9aadb-5bce-482b-871d-155083eeaf2b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:24.457729 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163816f-d22e-4089-b04e-86415b15565a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:24.458567 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163816f-d22e-4089-b04e-86415b15565a admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:24.459002 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163816f-d22e-4089-b04e-86415b15565a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:24.459515 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163816f-d22e-4089-b04e-86415b15565a admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:24.462407 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6163816f-d22e-4089-b04e-86415b15565a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:24.465605 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6163816f-d22e-4089-b04e-86415b15565a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:24.475434 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 985/1973] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:24 2022] PUT /identity/v3/projects/39b32a56c3694cc1b1f5027ae1b123d2/users/9476793c690d45019109159350a3b850/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:24.475434 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc52a6-63f1-4c9b-a722-f72e916660e7 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:24.480513 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f6cc52a6-63f1-4c9b-a722-f72e916660e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:24.582492 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc52a6-63f1-4c9b-a722-f72e916660e7 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:24.587616 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209bb281-26cb-4eba-b1b5-0ac3c8f16129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:24.588237 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209bb281-26cb-4eba-b1b5-0ac3c8f16129 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:24.588554 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209bb281-26cb-4eba-b1b5-0ac3c8f16129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:24.588845 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209bb281-26cb-4eba-b1b5-0ac3c8f16129 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:24.589469 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209bb281-26cb-4eba-b1b5-0ac3c8f16129 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:24.591451 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209bb281-26cb-4eba-b1b5-0ac3c8f16129 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:24.611927 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 989/1974] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:07:24 2022] POST /identity/v3/projects => generated 381 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:24.613157 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163816f-d22e-4089-b04e-86415b15565a admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:24.615021 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6163816f-d22e-4089-b04e-86415b15565a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:24.698423 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 986/1975] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:07:24 2022] POST /identity/v3/users => generated 289 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:24.699365 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209bb281-26cb-4eba-b1b5-0ac3c8f16129 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:24.700040 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163816f-d22e-4089-b04e-86415b15565a admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:24.702485 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-209bb281-26cb-4eba-b1b5-0ac3c8f16129 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:24.706504 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e05840-6517-4f28-a967-79e4d082c652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:24.707078 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e05840-6517-4f28-a967-79e4d082c652 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:24.707360 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e05840-6517-4f28-a967-79e4d082c652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:24.707754 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e05840-6517-4f28-a967-79e4d082c652 admin admin] PATH_INFO: `/v3/users/450925c27aa14dd4887cba54bab32a75` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:24.709029 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e05840-6517-4f28-a967-79e4d082c652 admin admin] RBAC: Authorizing `identity:delete_user(user_id=450925c27aa14dd4887cba54bab32a75)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:24.710999 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e05840-6517-4f28-a967-79e4d082c652 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:24.746029 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209bb281-26cb-4eba-b1b5-0ac3c8f16129 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:24.752350 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd1060-b393-4a87-9482-6b7dc5680656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:24.753017 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd1060-b393-4a87-9482-6b7dc5680656 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:24.753303 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd1060-b393-4a87-9482-6b7dc5680656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:24.753807 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd1060-b393-4a87-9482-6b7dc5680656 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:24.754648 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd1060-b393-4a87-9482-6b7dc5680656 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:24.756992 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd1060-b393-4a87-9482-6b7dc5680656 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:24.771571 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 987/1976] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:24 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:24.772546 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd1060-b393-4a87-9482-6b7dc5680656 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:24.786117 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd1060-b393-4a87-9482-6b7dc5680656 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:24.792530 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c901c3c-4196-4d1c-a60c-97b2253830dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:24.793161 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c901c3c-4196-4d1c-a60c-97b2253830dc admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:24.793564 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c901c3c-4196-4d1c-a60c-97b2253830dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:24.793918 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c901c3c-4196-4d1c-a60c-97b2253830dc admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:24.794574 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c901c3c-4196-4d1c-a60c-97b2253830dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:24.796718 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c901c3c-4196-4d1c-a60c-97b2253830dc admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:24.906983 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 988/1977] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:07:24 2022] POST /identity/v3/users => generated 297 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:24.909517 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c901c3c-4196-4d1c-a60c-97b2253830dc admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:24.922238 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c901c3c-4196-4d1c-a60c-97b2253830dc admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:24.928552 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09773140-ea6a-4fac-86ab-85f6af1e2b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:24.929300 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09773140-ea6a-4fac-86ab-85f6af1e2b3a admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:24.929689 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09773140-ea6a-4fac-86ab-85f6af1e2b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:24.930032 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09773140-ea6a-4fac-86ab-85f6af1e2b3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:24.930778 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09773140-ea6a-4fac-86ab-85f6af1e2b3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:24.932718 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09773140-ea6a-4fac-86ab-85f6af1e2b3a admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:24.945538 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 989/1978] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:24.946456 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09773140-ea6a-4fac-86ab-85f6af1e2b3a admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:24.957832 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-68e05840-6517-4f28-a967-79e4d082c652 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '450925c27aa14dd4887cba54bab32a75'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:24.959289 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-68e05840-6517-4f28-a967-79e4d082c652 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '450925c27aa14dd4887cba54bab32a75'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:24.962314 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09773140-ea6a-4fac-86ab-85f6af1e2b3a admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:24.968315 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8316256-41de-42b3-913c-f51a83504a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:24.969479 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8316256-41de-42b3-913c-f51a83504a0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:24.969776 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8316256-41de-42b3-913c-f51a83504a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:24.970091 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8316256-41de-42b3-913c-f51a83504a0b admin admin] PATH_INFO: `/v3/projects/39b32a56c3694cc1b1f5027ae1b123d2/users/9476793c690d45019109159350a3b850/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:24.971724 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-68e05840-6517-4f28-a967-79e4d082c652 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '450925c27aa14dd4887cba54bab32a75'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:24.972665 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8316256-41de-42b3-913c-f51a83504a0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=39b32a56c3694cc1b1f5027ae1b123d2, user_id=9476793c690d45019109159350a3b850, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:24.975704 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8316256-41de-42b3-913c-f51a83504a0b admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:24.996989 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 990/1979] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:24 2022] PUT /identity/v3/projects/39b32a56c3694cc1b1f5027ae1b123d2/users/9476793c690d45019109159350a3b850/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:24.996989 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8316256-41de-42b3-913c-f51a83504a0b admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:25.000542 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c8316256-41de-42b3-913c-f51a83504a0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:25.079867 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8316256-41de-42b3-913c-f51a83504a0b admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:25.087888 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c5f24-63f2-495a-bd30-326e99f092db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:25.092665 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-68e05840-6517-4f28-a967-79e4d082c652 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '450925c27aa14dd4887cba54bab32a75'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:25.092665 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-68e05840-6517-4f28-a967-79e4d082c652 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '450925c27aa14dd4887cba54bab32a75'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:25.093044 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c5f24-63f2-495a-bd30-326e99f092db admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:25.093044 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c5f24-63f2-495a-bd30-326e99f092db admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:25.093044 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c5f24-63f2-495a-bd30-326e99f092db admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:25.093044 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64c5f24-63f2-495a-bd30-326e99f092db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:25.093044 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64c5f24-63f2-495a-bd30-326e99f092db admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:25.136011 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 990/1980] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:07:24 2022] DELETE /identity/v3/users/450925c27aa14dd4887cba54bab32a75 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:25.142484 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e05840-6517-4f28-a967-79e4d082c652 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:25.144428 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 991/1981] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:24 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:25.156660 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d604a-bd71-4979-abe2-cb50b950d971 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:25.156660 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d604a-bd71-4979-abe2-cb50b950d971 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:25.156660 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d604a-bd71-4979-abe2-cb50b950d971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:25.157153 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-68e05840-6517-4f28-a967-79e4d082c652 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:25.185100 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-4e1d604a-bd71-4979-abe2-cb50b950d971 None None] MFA Rules not processed for user `9476793c690d45019109159350a3b850`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:25.186440 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-4e1d604a-bd71-4979-abe2-cb50b950d971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:25.225306 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:25.238341 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e05840-6517-4f28-a967-79e4d082c652 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:25.244246 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b99b6-be94-472f-a162-144dadd3ffad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:25.244955 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b99b6-be94-472f-a162-144dadd3ffad admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:25.245238 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b99b6-be94-472f-a162-144dadd3ffad admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:25.245514 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b99b6-be94-472f-a162-144dadd3ffad admin admin] PATH_INFO: `/v3/projects/1d143b484b354585b57992eb8f96576e/users/deb8f9e8938841d787442e8e46e62195/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:25.261954 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2b99b6-be94-472f-a162-144dadd3ffad admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d143b484b354585b57992eb8f96576e, user_id=deb8f9e8938841d787442e8e46e62195, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:25.264891 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2b99b6-be94-472f-a162-144dadd3ffad admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:25.273428 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 992/1982] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:25 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:25.274408 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d604a-bd71-4979-abe2-cb50b950d971 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:25.285625 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d604a-bd71-4979-abe2-cb50b950d971 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:25.290387 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca3b32-0d61-4035-8bd2-62844259acf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:25.290952 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca3b32-0d61-4035-8bd2-62844259acf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:25.291235 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca3b32-0d61-4035-8bd2-62844259acf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:25.291515 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca3b32-0d61-4035-8bd2-62844259acf1 admin admin] PATH_INFO: `/v3/projects/6f459dbad6394e49a1689bcd1766a5f8/users/d199b3de462a44959377abf51a60aa00/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:25.308084 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca3b32-0d61-4035-8bd2-62844259acf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f459dbad6394e49a1689bcd1766a5f8, user_id=d199b3de462a44959377abf51a60aa00, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:25.310490 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca3b32-0d61-4035-8bd2-62844259acf1 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:25.331088 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 991/1983] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:25 2022] PUT /identity/v3/projects/1d143b484b354585b57992eb8f96576e/users/deb8f9e8938841d787442e8e46e62195/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:25.332191 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b99b6-be94-472f-a162-144dadd3ffad admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:25.334223 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-eb2b99b6-be94-472f-a162-144dadd3ffad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:25.381470 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 993/1984] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:25 2022] PUT /identity/v3/projects/6f459dbad6394e49a1689bcd1766a5f8/users/d199b3de462a44959377abf51a60aa00/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:25.381470 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca3b32-0d61-4035-8bd2-62844259acf1 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:25.397183 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b99b6-be94-472f-a162-144dadd3ffad admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:25.454396 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a0e4a5-ab70-4900-9c94-cb85e1c1ed5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:25.455240 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a0e4a5-ab70-4900-9c94-cb85e1c1ed5f service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:25.455587 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a0e4a5-ab70-4900-9c94-cb85e1c1ed5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:25.455865 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a0e4a5-ab70-4900-9c94-cb85e1c1ed5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:25.469185 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:25.482681 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca3b32-0d61-4035-8bd2-62844259acf1 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:25.489256 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0318c7-cc76-48a6-9463-8b0765e16142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:25.490540 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0318c7-cc76-48a6-9463-8b0765e16142 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:25.490540 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0318c7-cc76-48a6-9463-8b0765e16142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:25.490973 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0318c7-cc76-48a6-9463-8b0765e16142 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:25.491765 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0318c7-cc76-48a6-9463-8b0765e16142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:25.494128 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0318c7-cc76-48a6-9463-8b0765e16142 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:25.513013 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 994/1985] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:25.515145 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0318c7-cc76-48a6-9463-8b0765e16142 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:25.525981 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a0e4a5-ab70-4900-9c94-cb85e1c1ed5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:25.527400 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a0e4a5-ab70-4900-9c94-cb85e1c1ed5f service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:25.533179 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0318c7-cc76-48a6-9463-8b0765e16142 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:25.552449 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 992/1986] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:25 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:07:25.552449 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a0e4a5-ab70-4900-9c94-cb85e1c1ed5f service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:25.555043 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ba9f0-3507-4abf-b5bc-2155bca2b519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:25.558610 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ba9f0-3507-4abf-b5bc-2155bca2b519 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:25.559447 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ba9f0-3507-4abf-b5bc-2155bca2b519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:25.560373 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ba9f0-3507-4abf-b5bc-2155bca2b519 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:25.576514 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a0e4a5-ab70-4900-9c94-cb85e1c1ed5f service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:25.641110 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ba9f0-3507-4abf-b5bc-2155bca2b519 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:25.641707 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0091537-f052-4dfe-87e4-ef04a6994f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:25.641707 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0091537-f052-4dfe-87e4-ef04a6994f91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:25.641707 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0091537-f052-4dfe-87e4-ef04a6994f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:25.641707 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0091537-f052-4dfe-87e4-ef04a6994f91 admin admin] PATH_INFO: `/v3/projects/1d143b484b354585b57992eb8f96576e/users/deb8f9e8938841d787442e8e46e62195/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:25.642071 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ba9f0-3507-4abf-b5bc-2155bca2b519 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:25.642164 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0091537-f052-4dfe-87e4-ef04a6994f91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d143b484b354585b57992eb8f96576e, user_id=deb8f9e8938841d787442e8e46e62195, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:25.642164 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0091537-f052-4dfe-87e4-ef04a6994f91 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:25.642335 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 995/1987] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:25.642425 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 993/1988] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:25 2022] PUT /identity/v3/projects/1d143b484b354585b57992eb8f96576e/users/deb8f9e8938841d787442e8e46e62195/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:25.642515 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ba9f0-3507-4abf-b5bc-2155bca2b519 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:25.648051 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3809c9-827e-49fa-ad8b-c2e61fd8deef None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:25.648549 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3809c9-827e-49fa-ad8b-c2e61fd8deef None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:25.648900 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3809c9-827e-49fa-ad8b-c2e61fd8deef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:25.655937 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ba9f0-3507-4abf-b5bc-2155bca2b519 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:25.692095 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-7e3809c9-827e-49fa-ad8b-c2e61fd8deef None None] MFA Rules not processed for user `deb8f9e8938841d787442e8e46e62195`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:25.693907 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7e3809c9-827e-49fa-ad8b-c2e61fd8deef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:25.802999 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 994/1989] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:25 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:07:25.819204 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3809c9-827e-49fa-ad8b-c2e61fd8deef None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:25.819204 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7e3809c9-827e-49fa-ad8b-c2e61fd8deef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:25.822534 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd20aa45-ba4d-4fe9-b12e-d319efc2bf62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:25.827079 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20aa45-ba4d-4fe9-b12e-d319efc2bf62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:25.827871 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20aa45-ba4d-4fe9-b12e-d319efc2bf62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:25.828166 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20aa45-ba4d-4fe9-b12e-d319efc2bf62 admin admin] PATH_INFO: `/v3/projects/6f459dbad6394e49a1689bcd1766a5f8/users/d199b3de462a44959377abf51a60aa00/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:25.831564 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd20aa45-ba4d-4fe9-b12e-d319efc2bf62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f459dbad6394e49a1689bcd1766a5f8, user_id=d199b3de462a44959377abf51a60aa00, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:25.834295 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd20aa45-ba4d-4fe9-b12e-d319efc2bf62 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:25.861160 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 996/1990] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:25 2022] PUT /identity/v3/projects/6f459dbad6394e49a1689bcd1766a5f8/users/d199b3de462a44959377abf51a60aa00/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:07:25.863192 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e1d99-c023-47bc-854e-d256e9602626 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:25.863693 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e1d99-c023-47bc-854e-d256e9602626 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:25.864135 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e1d99-c023-47bc-854e-d256e9602626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:25.939107 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-d96e1d99-c023-47bc-854e-d256e9602626 None None] MFA Rules not processed for user `d199b3de462a44959377abf51a60aa00`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:25.940490 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d96e1d99-c023-47bc-854e-d256e9602626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:25.986324 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3809c9-827e-49fa-ad8b-c2e61fd8deef None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:26.062602 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e83a7-28b6-4e6c-8cd7-527e80dd1281 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:26.063225 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e83a7-28b6-4e6c-8cd7-527e80dd1281 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:26.063899 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e83a7-28b6-4e6c-8cd7-527e80dd1281 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:26.064279 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e83a7-28b6-4e6c-8cd7-527e80dd1281 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:26.068224 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 997/1991] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:25 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:26.068224 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e1d99-c023-47bc-854e-d256e9602626 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:26.068224 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d96e1d99-c023-47bc-854e-d256e9602626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:26.188484 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e83a7-28b6-4e6c-8cd7-527e80dd1281 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:26.188716 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e1d99-c023-47bc-854e-d256e9602626 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:26.188816 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e83a7-28b6-4e6c-8cd7-527e80dd1281 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:26.188916 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7ea9d-7328-4845-8b20-2b46c34c3052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:26.188916 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7ea9d-7328-4845-8b20-2b46c34c3052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:26.188916 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7ea9d-7328-4845-8b20-2b46c34c3052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:26.188916 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7ea9d-7328-4845-8b20-2b46c34c3052 admin admin] PATH_INFO: `/v3/projects/9cb9ebee71d14175a203acb2b2251a36` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:26.189434 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a7ea9d-7328-4845-8b20-2b46c34c3052 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cb9ebee71d14175a203acb2b2251a36)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:26.189434 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a7ea9d-7328-4845-8b20-2b46c34c3052 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:26.189685 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 995/1992] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:25 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 01:07:26.189685 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e83a7-28b6-4e6c-8cd7-527e80dd1281 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:26.198561 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e83a7-28b6-4e6c-8cd7-527e80dd1281 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:26.202584 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea55ff-b047-4aa9-9b71-160463a702b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:26.206756 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea55ff-b047-4aa9-9b71-160463a702b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:26.206756 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea55ff-b047-4aa9-9b71-160463a702b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:26.206756 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea55ff-b047-4aa9-9b71-160463a702b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:26.217292 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea55ff-b047-4aa9-9b71-160463a702b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:26.221775 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea55ff-b047-4aa9-9b71-160463a702b9 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:26.258503 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 996/1993] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:26 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:07:26.332032 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-62a7ea9d-7328-4845-8b20-2b46c34c3052 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cb9ebee71d14175a203acb2b2251a36', 'request_id': 'req-62a7ea9d-7328-4845-8b20-2b46c34c3052'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:26.462226 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-62a7ea9d-7328-4845-8b20-2b46c34c3052 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cb9ebee71d14175a203acb2b2251a36', 'request_id': 'req-62a7ea9d-7328-4845-8b20-2b46c34c3052'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:26.471997 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-62a7ea9d-7328-4845-8b20-2b46c34c3052 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cb9ebee71d14175a203acb2b2251a36', 'request_id': 'req-62a7ea9d-7328-4845-8b20-2b46c34c3052'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:26.486779 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-62a7ea9d-7328-4845-8b20-2b46c34c3052 admin admin] The token cache is being invalidate because project 9cb9ebee71d14175a203acb2b2251a36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:07:26.487008 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-62a7ea9d-7328-4845-8b20-2b46c34c3052 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:26.487678 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-62a7ea9d-7328-4845-8b20-2b46c34c3052 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:26.491440 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 998/1994] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:07:26 2022] DELETE /identity/v3/projects/9cb9ebee71d14175a203acb2b2251a36 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:26.492525 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea55ff-b047-4aa9-9b71-160463a702b9 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:26.496308 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f6b37-1d21-42a1-b2f2-089d9bbcd4c8 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:26.496760 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f6b37-1d21-42a1-b2f2-089d9bbcd4c8 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:26.498596 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f6b37-1d21-42a1-b2f2-089d9bbcd4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:26.503626 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea55ff-b047-4aa9-9b71-160463a702b9 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:26.537850 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-a48f6b37-1d21-42a1-b2f2-089d9bbcd4c8 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:26.537850 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a48f6b37-1d21-42a1-b2f2-089d9bbcd4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:26.570715 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749b15a-099e-41ab-a6f1-ce8938249db2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:26.571617 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749b15a-099e-41ab-a6f1-ce8938249db2 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:26.572091 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749b15a-099e-41ab-a6f1-ce8938249db2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:26.572554 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749b15a-099e-41ab-a6f1-ce8938249db2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:26.576502 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2749b15a-099e-41ab-a6f1-ce8938249db2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:26.638876 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 999/1995] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:26.641747 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d166c24-38b6-4a81-a285-12933cbf8748 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:26.645810 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d166c24-38b6-4a81-a285-12933cbf8748 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:26.645810 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d166c24-38b6-4a81-a285-12933cbf8748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:26.674727 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-0d166c24-38b6-4a81-a285-12933cbf8748 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:26.676106 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-0d166c24-38b6-4a81-a285-12933cbf8748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:26.688994 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1000/1996] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:26.693409 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d166c24-38b6-4a81-a285-12933cbf8748 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:26.703365 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2749b15a-099e-41ab-a6f1-ce8938249db2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:26.706892 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d166c24-38b6-4a81-a285-12933cbf8748 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:26.713056 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2749b15a-099e-41ab-a6f1-ce8938249db2 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:26.716582 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc913bf9-c04c-4f98-829e-196f0bb2ee73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:26.716582 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc913bf9-c04c-4f98-829e-196f0bb2ee73 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:26.716582 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc913bf9-c04c-4f98-829e-196f0bb2ee73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:26.716582 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc913bf9-c04c-4f98-829e-196f0bb2ee73 admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:26.716991 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc913bf9-c04c-4f98-829e-196f0bb2ee73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:07:26.716991 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc913bf9-c04c-4f98-829e-196f0bb2ee73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:26.718296 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc913bf9-c04c-4f98-829e-196f0bb2ee73 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:26.730822 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 997/1997] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:07:26.735991 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1001/1998] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:07:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:07:26.737330 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749b15a-099e-41ab-a6f1-ce8938249db2 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:26.749785 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749b15a-099e-41ab-a6f1-ce8938249db2 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:26.757676 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5144c1e-4984-4466-93cf-c6309b964e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:26.758591 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5144c1e-4984-4466-93cf-c6309b964e5d admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:26.759042 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5144c1e-4984-4466-93cf-c6309b964e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:26.759444 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5144c1e-4984-4466-93cf-c6309b964e5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:26.760331 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5144c1e-4984-4466-93cf-c6309b964e5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:26.762631 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5144c1e-4984-4466-93cf-c6309b964e5d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:26.857676 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 998/1999] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:07:26 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:26.862471 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc913bf9-c04c-4f98-829e-196f0bb2ee73 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:26.872581 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc913bf9-c04c-4f98-829e-196f0bb2ee73 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:26.933178 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28ba02-e994-4dbd-b93d-d528ab340a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:26.933946 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28ba02-e994-4dbd-b93d-d528ab340a5a admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:26.934300 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28ba02-e994-4dbd-b93d-d528ab340a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:26.934666 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28ba02-e994-4dbd-b93d-d528ab340a5a admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:26.935370 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d28ba02-e994-4dbd-b93d-d528ab340a5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:26.937421 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d28ba02-e994-4dbd-b93d-d528ab340a5a admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:27.029877 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1002/2000] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:07:26 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:27.034485 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5144c1e-4984-4466-93cf-c6309b964e5d admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:27.050445 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5144c1e-4984-4466-93cf-c6309b964e5d admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:27.052213 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:27.056099 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a7877-2d4d-47c7-af5a-9d4f9e87a5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:27.056764 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a7877-2d4d-47c7-af5a-9d4f9e87a5fe admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:27.057046 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a7877-2d4d-47c7-af5a-9d4f9e87a5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:27.057328 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a7877-2d4d-47c7-af5a-9d4f9e87a5fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:27.057968 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a7877-2d4d-47c7-af5a-9d4f9e87a5fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:27.059609 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a7877-2d4d-47c7-af5a-9d4f9e87a5fe admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:27.071846 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 999/2001] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 01:07:27.076078 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28ba02-e994-4dbd-b93d-d528ab340a5a admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:27.089212 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28ba02-e994-4dbd-b93d-d528ab340a5a admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:27.094442 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ec4d9-6261-44db-873d-1e6d1f0e983b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:27.095108 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ec4d9-6261-44db-873d-1e6d1f0e983b admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:27.095721 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ec4d9-6261-44db-873d-1e6d1f0e983b admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:27.095721 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ec4d9-6261-44db-873d-1e6d1f0e983b admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:27.096663 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ec4d9-6261-44db-873d-1e6d1f0e983b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:27.098985 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ec4d9-6261-44db-873d-1e6d1f0e983b admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:27.113970 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1003/2002] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 01:07:27.118275 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a7877-2d4d-47c7-af5a-9d4f9e87a5fe admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:27.129773 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a7877-2d4d-47c7-af5a-9d4f9e87a5fe admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:27.134420 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1747f0-9bc6-48cc-827c-e3ac5c9c4bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:27.135070 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1747f0-9bc6-48cc-827c-e3ac5c9c4bbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:27.135350 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1747f0-9bc6-48cc-827c-e3ac5c9c4bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:27.135624 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1747f0-9bc6-48cc-827c-e3ac5c9c4bbb admin admin] PATH_INFO: `/v3/projects/31cd0563b999440fbe93f7d4bce54eca/users/eb652004dbcf419b81e15010c2ea76a1/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:27.157083 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1747f0-9bc6-48cc-827c-e3ac5c9c4bbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=31cd0563b999440fbe93f7d4bce54eca, user_id=eb652004dbcf419b81e15010c2ea76a1, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:27.157947 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1747f0-9bc6-48cc-827c-e3ac5c9c4bbb admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:27.174578 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ec4d9-6261-44db-873d-1e6d1f0e983b admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:27.176821 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3d4ec4d9-6261-44db-873d-1e6d1f0e983b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:27.245330 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1000/2003] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:27 2022] PUT /identity/v3/projects/31cd0563b999440fbe93f7d4bce54eca/users/eb652004dbcf419b81e15010c2ea76a1/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:27.256839 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b6106-4e6e-4617-94b7-da6d984f5b1d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:27.257131 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b6106-4e6e-4617-94b7-da6d984f5b1d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:27.257442 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b6106-4e6e-4617-94b7-da6d984f5b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:27.275218 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ec4d9-6261-44db-873d-1e6d1f0e983b admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:27.300858 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-9d1b6106-4e6e-4617-94b7-da6d984f5b1d None None] MFA Rules not processed for user `eb652004dbcf419b81e15010c2ea76a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:27.303820 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9d1b6106-4e6e-4617-94b7-da6d984f5b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:27.326858 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:27.338662 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497cac1-d360-4fa6-a458-cba45acf3e71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:27.339929 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497cac1-d360-4fa6-a458-cba45acf3e71 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:27.340528 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497cac1-d360-4fa6-a458-cba45acf3e71 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:27.341064 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497cac1-d360-4fa6-a458-cba45acf3e71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:27.343507 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-4497cac1-d360-4fa6-a458-cba45acf3e71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:27.404074 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1001/2004] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:07:27.410316 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b6106-4e6e-4617-94b7-da6d984f5b1d None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:27.433326 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4497cac1-d360-4fa6-a458-cba45acf3e71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:27.434928 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4497cac1-d360-4fa6-a458-cba45acf3e71 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:27.446579 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1004/2005] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:27 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 01:07:27.495832 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b6106-4e6e-4617-94b7-da6d984f5b1d None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:27.502481 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa7cd2-d3ee-4171-8744-bb7bf7b68047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:27.503321 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa7cd2-d3ee-4171-8744-bb7bf7b68047 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:27.503703 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa7cd2-d3ee-4171-8744-bb7bf7b68047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:27.504078 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa7cd2-d3ee-4171-8744-bb7bf7b68047 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:27.505133 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aa7cd2-d3ee-4171-8744-bb7bf7b68047 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:27.511844 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aa7cd2-d3ee-4171-8744-bb7bf7b68047 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:27.590676 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497cac1-d360-4fa6-a458-cba45acf3e71 service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:27.592265 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-4497cac1-d360-4fa6-a458-cba45acf3e71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:27.609246 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497cac1-d360-4fa6-a458-cba45acf3e71 service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:27.614368 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff656e-fbe7-4f23-973f-21e721d17650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:27.615345 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff656e-fbe7-4f23-973f-21e721d17650 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:27.615345 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff656e-fbe7-4f23-973f-21e721d17650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:27.615562 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff656e-fbe7-4f23-973f-21e721d17650 admin admin] PATH_INFO: `/v3/users/aebea04dcc6044978b6915dbcabcb01a` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:27.616811 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff656e-fbe7-4f23-973f-21e721d17650 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aebea04dcc6044978b6915dbcabcb01a)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:27.627630 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1002/2006] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:07:27 2022] POST /identity/v3/projects => generated 383 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:27.627886 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff656e-fbe7-4f23-973f-21e721d17650 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:27.631185 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa7cd2-d3ee-4171-8744-bb7bf7b68047 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:27.734288 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa7cd2-d3ee-4171-8744-bb7bf7b68047 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:27.740003 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ca6ec-0767-4cc4-9747-0c36108eab4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:27.740652 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ca6ec-0767-4cc4-9747-0c36108eab4a admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:27.741014 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ca6ec-0767-4cc4-9747-0c36108eab4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:27.741240 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ca6ec-0767-4cc4-9747-0c36108eab4a admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:27.741869 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036ca6ec-0767-4cc4-9747-0c36108eab4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:27.743783 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036ca6ec-0767-4cc4-9747-0c36108eab4a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:27.854395 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1003/2007] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:07:27 2022] POST /identity/v3/users => generated 298 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:07:27.855461 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ca6ec-0767-4cc4-9747-0c36108eab4a admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:27.865634 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ca6ec-0767-4cc4-9747-0c36108eab4a admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:27.871513 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f123049-7986-423e-9cef-a8c6d9cd705b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:27.872213 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f123049-7986-423e-9cef-a8c6d9cd705b admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:27.872746 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f123049-7986-423e-9cef-a8c6d9cd705b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:27.872958 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f123049-7986-423e-9cef-a8c6d9cd705b admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:27.873595 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f123049-7986-423e-9cef-a8c6d9cd705b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:27.875240 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f123049-7986-423e-9cef-a8c6d9cd705b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:27.887272 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1004/2008] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:27.887881 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-93ff656e-fbe7-4f23-973f-21e721d17650 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aebea04dcc6044978b6915dbcabcb01a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:27.889101 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-93ff656e-fbe7-4f23-973f-21e721d17650 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aebea04dcc6044978b6915dbcabcb01a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:27.891743 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f123049-7986-423e-9cef-a8c6d9cd705b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:27.894445 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7f123049-7986-423e-9cef-a8c6d9cd705b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:27.907761 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f123049-7986-423e-9cef-a8c6d9cd705b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:27.912869 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee308d2-15a4-4832-9e47-53c045e6675e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:27.913523 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee308d2-15a4-4832-9e47-53c045e6675e admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:27.913802 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee308d2-15a4-4832-9e47-53c045e6675e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:27.914106 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee308d2-15a4-4832-9e47-53c045e6675e admin admin] PATH_INFO: `/v3/projects/ab06f0b03ab440b7bfe73e8a7d7b606b/users/707e05437ff142fd836d11ad9b3fbe8f/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:27.928670 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee308d2-15a4-4832-9e47-53c045e6675e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab06f0b03ab440b7bfe73e8a7d7b606b, user_id=707e05437ff142fd836d11ad9b3fbe8f, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:27.931316 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee308d2-15a4-4832-9e47-53c045e6675e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:27.979057 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-93ff656e-fbe7-4f23-973f-21e721d17650 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aebea04dcc6044978b6915dbcabcb01a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:27.979664 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-93ff656e-fbe7-4f23-973f-21e721d17650 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aebea04dcc6044978b6915dbcabcb01a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:27.992001 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-93ff656e-fbe7-4f23-973f-21e721d17650 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aebea04dcc6044978b6915dbcabcb01a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:28.004607 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1005/2009] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:07:27 2022] DELETE /identity/v3/users/aebea04dcc6044978b6915dbcabcb01a => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:28.009222 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784bedca-cd14-4c37-bd08-418d35b38343 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:28.009875 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784bedca-cd14-4c37-bd08-418d35b38343 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:28.009875 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784bedca-cd14-4c37-bd08-418d35b38343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:28.021794 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1005/2010] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:27 2022] PUT /identity/v3/projects/ab06f0b03ab440b7bfe73e8a7d7b606b/users/707e05437ff142fd836d11ad9b3fbe8f/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:28.024556 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee308d2-15a4-4832-9e47-53c045e6675e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:28.031409 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-784bedca-cd14-4c37-bd08-418d35b38343 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:28.033045 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-784bedca-cd14-4c37-bd08-418d35b38343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:28.108654 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1006/2011] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:07:28.126647 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784bedca-cd14-4c37-bd08-418d35b38343 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:28.132293 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-784bedca-cd14-4c37-bd08-418d35b38343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:28.148783 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee308d2-15a4-4832-9e47-53c045e6675e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:28.153707 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521220bc-f2d5-4410-9b77-5ce103548f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:28.154328 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521220bc-f2d5-4410-9b77-5ce103548f87 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:28.154610 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521220bc-f2d5-4410-9b77-5ce103548f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:28.154883 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521220bc-f2d5-4410-9b77-5ce103548f87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:28.155523 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521220bc-f2d5-4410-9b77-5ce103548f87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:28.158265 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521220bc-f2d5-4410-9b77-5ce103548f87 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:28.171300 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1006/2012] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:28.172357 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521220bc-f2d5-4410-9b77-5ce103548f87 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:28.190104 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521220bc-f2d5-4410-9b77-5ce103548f87 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:28.202591 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dafe9f0-034f-447e-86dd-ba7289479157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:28.203339 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafe9f0-034f-447e-86dd-ba7289479157 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:28.203626 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafe9f0-034f-447e-86dd-ba7289479157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:28.203908 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafe9f0-034f-447e-86dd-ba7289479157 admin admin] PATH_INFO: `/v3/projects/ab06f0b03ab440b7bfe73e8a7d7b606b/users/707e05437ff142fd836d11ad9b3fbe8f/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:28.206882 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dafe9f0-034f-447e-86dd-ba7289479157 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab06f0b03ab440b7bfe73e8a7d7b606b, user_id=707e05437ff142fd836d11ad9b3fbe8f, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:28.209916 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dafe9f0-034f-447e-86dd-ba7289479157 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:28.226219 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784bedca-cd14-4c37-bd08-418d35b38343 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:28.233775 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1007/2013] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:28 2022] PUT /identity/v3/projects/ab06f0b03ab440b7bfe73e8a7d7b606b/users/707e05437ff142fd836d11ad9b3fbe8f/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:07:28.245121 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e111d-d9f5-4e5a-8ef6-d97ec97f9745 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:28.245800 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e111d-d9f5-4e5a-8ef6-d97ec97f9745 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:28.246080 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e111d-d9f5-4e5a-8ef6-d97ec97f9745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:28.292868 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-bb9e111d-d9f5-4e5a-8ef6-d97ec97f9745 None None] MFA Rules not processed for user `707e05437ff142fd836d11ad9b3fbe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:28.294825 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bb9e111d-d9f5-4e5a-8ef6-d97ec97f9745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:28.298499 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e081942-f944-4451-afc5-e1333a501c4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:28.299348 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e081942-f944-4451-afc5-e1333a501c4b service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:28.299469 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e081942-f944-4451-afc5-e1333a501c4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:28.299700 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e081942-f944-4451-afc5-e1333a501c4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:28.375216 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e081942-f944-4451-afc5-e1333a501c4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:28.375504 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1008/2014] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:28 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:28.377581 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e081942-f944-4451-afc5-e1333a501c4b service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:28.386510 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9e111d-d9f5-4e5a-8ef6-d97ec97f9745 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:28.387573 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1007/2015] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 01:07:28.387878 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bb9e111d-d9f5-4e5a-8ef6-d97ec97f9745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:28.463319 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9e111d-d9f5-4e5a-8ef6-d97ec97f9745 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:28.467492 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc3d99-9c15-4516-9838-c4dface4981e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:28.468210 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc3d99-9c15-4516-9838-c4dface4981e service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:28.468554 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc3d99-9c15-4516-9838-c4dface4981e service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:28.468829 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc3d99-9c15-4516-9838-c4dface4981e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:28.479991 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc3d99-9c15-4516-9838-c4dface4981e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:28.481921 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc3d99-9c15-4516-9838-c4dface4981e service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:28.492635 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1009/2016] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:28 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 01:07:28.648779 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:28.949697 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e081942-f944-4451-afc5-e1333a501c4b service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:28.951165 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-1e081942-f944-4451-afc5-e1333a501c4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:28.962546 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:28.975253 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e081942-f944-4451-afc5-e1333a501c4b service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:28.984945 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833dd2a3-147d-43c8-babf-44cbe58a89d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:28.986563 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833dd2a3-147d-43c8-babf-44cbe58a89d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:28.987576 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833dd2a3-147d-43c8-babf-44cbe58a89d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:28.987985 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833dd2a3-147d-43c8-babf-44cbe58a89d8 admin admin] PATH_INFO: `/v3/projects/c30877420157436cb3c46ba6c9a61ad9` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:28.993300 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833dd2a3-147d-43c8-babf-44cbe58a89d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c30877420157436cb3c46ba6c9a61ad9)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:28.995249 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833dd2a3-147d-43c8-babf-44cbe58a89d8 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:29.146431 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-833dd2a3-147d-43c8-babf-44cbe58a89d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c30877420157436cb3c46ba6c9a61ad9', 'request_id': 'req-833dd2a3-147d-43c8-babf-44cbe58a89d8'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:29.213439 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-833dd2a3-147d-43c8-babf-44cbe58a89d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c30877420157436cb3c46ba6c9a61ad9', 'request_id': 'req-833dd2a3-147d-43c8-babf-44cbe58a89d8'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:29.231560 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-833dd2a3-147d-43c8-babf-44cbe58a89d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c30877420157436cb3c46ba6c9a61ad9', 'request_id': 'req-833dd2a3-147d-43c8-babf-44cbe58a89d8'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:29.249292 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-833dd2a3-147d-43c8-babf-44cbe58a89d8 admin admin] The token cache is being invalidate because project c30877420157436cb3c46ba6c9a61ad9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:07:29.249586 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-833dd2a3-147d-43c8-babf-44cbe58a89d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:29.250249 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-833dd2a3-147d-43c8-babf-44cbe58a89d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:29.256042 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1008/2017] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:07:28 2022] DELETE /identity/v3/projects/c30877420157436cb3c46ba6c9a61ad9 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:29.257510 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc3d99-9c15-4516-9838-c4dface4981e service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:29.269544 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc3d99-9c15-4516-9838-c4dface4981e service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:29.340719 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b1d83-cd49-40e6-a797-efb9b4eccd46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:29.340719 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b1d83-cd49-40e6-a797-efb9b4eccd46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:29.340719 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b1d83-cd49-40e6-a797-efb9b4eccd46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:29.340719 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b1d83-cd49-40e6-a797-efb9b4eccd46 admin admin] PATH_INFO: `/v3/users/d199b3de462a44959377abf51a60aa00` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:29.341634 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b1d83-cd49-40e6-a797-efb9b4eccd46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d199b3de462a44959377abf51a60aa00)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:29.343713 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b1d83-cd49-40e6-a797-efb9b4eccd46 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:29.575241 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-624b1d83-cd49-40e6-a797-efb9b4eccd46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd199b3de462a44959377abf51a60aa00'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:29.575912 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-624b1d83-cd49-40e6-a797-efb9b4eccd46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd199b3de462a44959377abf51a60aa00'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:29.583306 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-624b1d83-cd49-40e6-a797-efb9b4eccd46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd199b3de462a44959377abf51a60aa00'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:29.668436 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-624b1d83-cd49-40e6-a797-efb9b4eccd46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd199b3de462a44959377abf51a60aa00'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:29.669294 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-624b1d83-cd49-40e6-a797-efb9b4eccd46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd199b3de462a44959377abf51a60aa00'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:29.696645 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1010/2018] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:07:29 2022] DELETE /identity/v3/users/d199b3de462a44959377abf51a60aa00 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:07:29.912158 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833dd2a3-147d-43c8-babf-44cbe58a89d8 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:29.913621 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-833dd2a3-147d-43c8-babf-44cbe58a89d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:29.926950 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833dd2a3-147d-43c8-babf-44cbe58a89d8 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:29.992381 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0382a8-ccc9-42dc-896c-30e20994e1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:29.993899 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0382a8-ccc9-42dc-896c-30e20994e1e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:29.994333 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0382a8-ccc9-42dc-896c-30e20994e1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:29.994756 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0382a8-ccc9-42dc-896c-30e20994e1e5 admin admin] PATH_INFO: `/v3/users/eb652004dbcf419b81e15010c2ea76a1` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:29.996057 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0382a8-ccc9-42dc-896c-30e20994e1e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb652004dbcf419b81e15010c2ea76a1)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:29.998418 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0382a8-ccc9-42dc-896c-30e20994e1e5 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:30.131174 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:30.246756 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b1d83-cd49-40e6-a797-efb9b4eccd46 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:30.254290 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-624b1d83-cd49-40e6-a797-efb9b4eccd46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:30.275584 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ae0382a8-ccc9-42dc-896c-30e20994e1e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb652004dbcf419b81e15010c2ea76a1'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:30.276567 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ae0382a8-ccc9-42dc-896c-30e20994e1e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb652004dbcf419b81e15010c2ea76a1'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:30.281959 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b1d83-cd49-40e6-a797-efb9b4eccd46 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:30.288657 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495fe46-bdea-462d-be62-9f29927cc975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:30.289661 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495fe46-bdea-462d-be62-9f29927cc975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:30.290141 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495fe46-bdea-462d-be62-9f29927cc975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:30.290619 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495fe46-bdea-462d-be62-9f29927cc975 admin admin] PATH_INFO: `/v3/projects/6f459dbad6394e49a1689bcd1766a5f8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:30.293036 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8495fe46-bdea-462d-be62-9f29927cc975 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f459dbad6394e49a1689bcd1766a5f8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:30.295410 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8495fe46-bdea-462d-be62-9f29927cc975 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:30.423020 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ae0382a8-ccc9-42dc-896c-30e20994e1e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb652004dbcf419b81e15010c2ea76a1'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:30.424162 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ae0382a8-ccc9-42dc-896c-30e20994e1e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb652004dbcf419b81e15010c2ea76a1'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:30.439039 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ae0382a8-ccc9-42dc-896c-30e20994e1e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb652004dbcf419b81e15010c2ea76a1'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:30.450115 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8495fe46-bdea-462d-be62-9f29927cc975 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f459dbad6394e49a1689bcd1766a5f8', 'request_id': 'req-8495fe46-bdea-462d-be62-9f29927cc975'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:30.452664 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1009/2019] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:07:29 2022] DELETE /identity/v3/users/eb652004dbcf419b81e15010c2ea76a1 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:30.457532 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69338c0-b999-46f4-8031-cf2b28de2420 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:30.457918 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69338c0-b999-46f4-8031-cf2b28de2420 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:30.458200 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69338c0-b999-46f4-8031-cf2b28de2420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:30.478223 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-d69338c0-b999-46f4-8031-cf2b28de2420 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:30.479598 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d69338c0-b999-46f4-8031-cf2b28de2420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:30.525003 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8495fe46-bdea-462d-be62-9f29927cc975 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f459dbad6394e49a1689bcd1766a5f8', 'request_id': 'req-8495fe46-bdea-462d-be62-9f29927cc975'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:30.537780 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8495fe46-bdea-462d-be62-9f29927cc975 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f459dbad6394e49a1689bcd1766a5f8', 'request_id': 'req-8495fe46-bdea-462d-be62-9f29927cc975'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:30.544169 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8495fe46-bdea-462d-be62-9f29927cc975 admin admin] The token cache is being invalidate because project 6f459dbad6394e49a1689bcd1766a5f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:07:30.544483 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8495fe46-bdea-462d-be62-9f29927cc975 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:30.545138 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8495fe46-bdea-462d-be62-9f29927cc975 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:30.546939 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1011/2020] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:07:30 2022] DELETE /identity/v3/projects/6f459dbad6394e49a1689bcd1766a5f8 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:30.550634 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1010/2021] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:30.550776 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495fe46-bdea-462d-be62-9f29927cc975 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:30.551833 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6152239-e232-4bca-aa45-4406b5a28c58 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:30.552131 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6152239-e232-4bca-aa45-4406b5a28c58 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:30.552408 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6152239-e232-4bca-aa45-4406b5a28c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:30.557223 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8495fe46-bdea-462d-be62-9f29927cc975 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:30.573808 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-f6152239-e232-4bca-aa45-4406b5a28c58 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:30.575084 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f6152239-e232-4bca-aa45-4406b5a28c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:30.592098 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1011/2022] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:07:30.593482 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6152239-e232-4bca-aa45-4406b5a28c58 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:30.603190 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f6152239-e232-4bca-aa45-4406b5a28c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:30.611676 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:30.645035 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495fe46-bdea-462d-be62-9f29927cc975 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:30.651931 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1550c7-19c0-41c4-94a4-71903c226bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:30.652826 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1550c7-19c0-41c4-94a4-71903c226bae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:30.653439 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1550c7-19c0-41c4-94a4-71903c226bae admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:30.654005 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1550c7-19c0-41c4-94a4-71903c226bae admin admin] PATH_INFO: `/v3/users/926f67677d0741f28736021f0ed52a85` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:30.655737 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1550c7-19c0-41c4-94a4-71903c226bae admin admin] RBAC: Authorizing `identity:delete_user(user_id=926f67677d0741f28736021f0ed52a85)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:30.658223 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1550c7-19c0-41c4-94a4-71903c226bae admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:30.709686 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6152239-e232-4bca-aa45-4406b5a28c58 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:30.718913 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd8146-62bd-4a14-8fe7-2e891aa28823 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:30.719598 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd8146-62bd-4a14-8fe7-2e891aa28823 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:30.719913 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd8146-62bd-4a14-8fe7-2e891aa28823 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:30.720359 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd8146-62bd-4a14-8fe7-2e891aa28823 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:30.724060 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8fdd8146-62bd-4a14-8fe7-2e891aa28823 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:30.737929 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd8146-62bd-4a14-8fe7-2e891aa28823 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:30.741162 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd8146-62bd-4a14-8fe7-2e891aa28823 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:30.750870 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1012/2023] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:07:30.752140 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7538e0d-5009-40e0-a6a5-8c40a2bf7ed6 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:30.752628 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7538e0d-5009-40e0-a6a5-8c40a2bf7ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:30.752990 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7538e0d-5009-40e0-a6a5-8c40a2bf7ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:30.802735 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-a7538e0d-5009-40e0-a6a5-8c40a2bf7ed6 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:30.804179 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a7538e0d-5009-40e0-a6a5-8c40a2bf7ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:30.818571 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1013/2024] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:30.819839 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7538e0d-5009-40e0-a6a5-8c40a2bf7ed6 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:30.835295 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7538e0d-5009-40e0-a6a5-8c40a2bf7ed6 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:30.845204 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62787106-4f0b-49ee-b6dc-66b0b2ce3dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:30.846524 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62787106-4f0b-49ee-b6dc-66b0b2ce3dbb admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:30.846899 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62787106-4f0b-49ee-b6dc-66b0b2ce3dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:30.847517 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62787106-4f0b-49ee-b6dc-66b0b2ce3dbb admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:30.848377 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62787106-4f0b-49ee-b6dc-66b0b2ce3dbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:07:30.848950 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62787106-4f0b-49ee-b6dc-66b0b2ce3dbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:30.850792 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62787106-4f0b-49ee-b6dc-66b0b2ce3dbb admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:30.874672 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1014/2025] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:07:30.875666 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62787106-4f0b-49ee-b6dc-66b0b2ce3dbb admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:30.887219 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62787106-4f0b-49ee-b6dc-66b0b2ce3dbb admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:30.893551 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e67785e-d164-439e-ae35-a271a337d5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:30.894607 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e67785e-d164-439e-ae35-a271a337d5f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:30.897888 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e67785e-d164-439e-ae35-a271a337d5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:30.897888 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e67785e-d164-439e-ae35-a271a337d5f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:30.897888 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e67785e-d164-439e-ae35-a271a337d5f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:30.898885 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e67785e-d164-439e-ae35-a271a337d5f9 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:30.908379 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2d1550c7-19c0-41c4-94a4-71903c226bae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '926f67677d0741f28736021f0ed52a85'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:30.910043 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2d1550c7-19c0-41c4-94a4-71903c226bae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '926f67677d0741f28736021f0ed52a85'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:30.921976 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2d1550c7-19c0-41c4-94a4-71903c226bae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '926f67677d0741f28736021f0ed52a85'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:31.034026 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2d1550c7-19c0-41c4-94a4-71903c226bae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '926f67677d0741f28736021f0ed52a85'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:31.035130 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2d1550c7-19c0-41c4-94a4-71903c226bae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '926f67677d0741f28736021f0ed52a85'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:31.043901 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1015/2026] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:07:30 2022] POST /identity/v3/projects => generated 385 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:31.046700 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e67785e-d164-439e-ae35-a271a337d5f9 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:31.048982 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5e67785e-d164-439e-ae35-a271a337d5f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:31.059503 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1012/2027] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:07:30 2022] DELETE /identity/v3/users/926f67677d0741f28736021f0ed52a85 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:31.063688 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1fb648-f486-4450-83ea-d7ce1d253b2d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:31.064058 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1fb648-f486-4450-83ea-d7ce1d253b2d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:31.064335 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1fb648-f486-4450-83ea-d7ce1d253b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:31.092242 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-3d1fb648-f486-4450-83ea-d7ce1d253b2d None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:31.093603 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3d1fb648-f486-4450-83ea-d7ce1d253b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:31.139728 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e67785e-d164-439e-ae35-a271a337d5f9 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:31.226710 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31343190-b4b3-4b9a-89ca-8867fb2d8ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:31.227533 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31343190-b4b3-4b9a-89ca-8867fb2d8ffa admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:31.227664 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1013/2028] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:31.227934 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31343190-b4b3-4b9a-89ca-8867fb2d8ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:31.228883 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31343190-b4b3-4b9a-89ca-8867fb2d8ffa admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:31.229909 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31343190-b4b3-4b9a-89ca-8867fb2d8ffa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:31.236646 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31343190-b4b3-4b9a-89ca-8867fb2d8ffa admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:31.242192 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1fb648-f486-4450-83ea-d7ce1d253b2d None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:31.244616 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3d1fb648-f486-4450-83ea-d7ce1d253b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:31.311298 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1fb648-f486-4450-83ea-d7ce1d253b2d None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:31.315060 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b8beea-8b24-42fe-ba14-41e74e0397de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:31.315863 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8beea-8b24-42fe-ba14-41e74e0397de service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:31.316144 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8beea-8b24-42fe-ba14-41e74e0397de service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:31.316414 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8beea-8b24-42fe-ba14-41e74e0397de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:31.328443 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b8beea-8b24-42fe-ba14-41e74e0397de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:31.332025 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b8beea-8b24-42fe-ba14-41e74e0397de service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:31.341552 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1014/2029] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:07:31.390402 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1016/2030] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:07:31 2022] POST /identity/v3/users => generated 293 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:07:31.395202 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b8beea-8b24-42fe-ba14-41e74e0397de service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:31.400863 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31343190-b4b3-4b9a-89ca-8867fb2d8ffa admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:31.402925 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-31343190-b4b3-4b9a-89ca-8867fb2d8ffa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:31.417660 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b8beea-8b24-42fe-ba14-41e74e0397de service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:31.426367 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f021344-3c80-4c14-a3bf-c820ee240c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:31.427339 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f021344-3c80-4c14-a3bf-c820ee240c45 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:31.427788 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f021344-3c80-4c14-a3bf-c820ee240c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:31.437597 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f021344-3c80-4c14-a3bf-c820ee240c45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:31.437597 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f021344-3c80-4c14-a3bf-c820ee240c45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:31.437597 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f021344-3c80-4c14-a3bf-c820ee240c45 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:31.437860 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31343190-b4b3-4b9a-89ca-8867fb2d8ffa admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:31.443504 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb67122-95a5-4505-8792-84e270902c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:31.444381 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb67122-95a5-4505-8792-84e270902c5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:31.453047 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb67122-95a5-4505-8792-84e270902c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:31.453172 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb67122-95a5-4505-8792-84e270902c5d admin admin] PATH_INFO: `/v3/projects/31cd0563b999440fbe93f7d4bce54eca` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:31.459597 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb67122-95a5-4505-8792-84e270902c5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=31cd0563b999440fbe93f7d4bce54eca)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:31.461566 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb67122-95a5-4505-8792-84e270902c5d admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:31.467767 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1015/2031] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:31.471816 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f021344-3c80-4c14-a3bf-c820ee240c45 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:31.501515 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f021344-3c80-4c14-a3bf-c820ee240c45 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:31.509774 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c24151-6f99-4624-bb4d-7d8400aaa304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:31.510606 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c24151-6f99-4624-bb4d-7d8400aaa304 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:31.510984 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c24151-6f99-4624-bb4d-7d8400aaa304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:31.511858 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c24151-6f99-4624-bb4d-7d8400aaa304 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:31.512762 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c24151-6f99-4624-bb4d-7d8400aaa304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:31.515815 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c24151-6f99-4624-bb4d-7d8400aaa304 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:31.532510 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1016/2032] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:31.535191 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c24151-6f99-4624-bb4d-7d8400aaa304 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:31.562945 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c24151-6f99-4624-bb4d-7d8400aaa304 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:31.578391 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d1b99-53b3-45fc-9be1-8e561880c23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:31.580225 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d1b99-53b3-45fc-9be1-8e561880c23a admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:31.581362 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d1b99-53b3-45fc-9be1-8e561880c23a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:31.581663 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d1b99-53b3-45fc-9be1-8e561880c23a admin admin] PATH_INFO: `/v3/projects/1c9bcb20d46a4b26a9df04a0467dc063/users/74e7b6f4cd47468e9e5ea62befe4abf4/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:31.602032 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d1b99-53b3-45fc-9be1-8e561880c23a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c9bcb20d46a4b26a9df04a0467dc063, user_id=74e7b6f4cd47468e9e5ea62befe4abf4, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:31.605576 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d1b99-53b3-45fc-9be1-8e561880c23a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:31.617517 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-5cb67122-95a5-4505-8792-84e270902c5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31cd0563b999440fbe93f7d4bce54eca', 'request_id': 'req-5cb67122-95a5-4505-8792-84e270902c5d'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:31.699084 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1017/2033] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:31 2022] PUT /identity/v3/projects/1c9bcb20d46a4b26a9df04a0467dc063/users/74e7b6f4cd47468e9e5ea62befe4abf4/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:07:31.710938 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f638770d-2fd4-4d25-be2a-f3edab304b59 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:31.711127 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f638770d-2fd4-4d25-be2a-f3edab304b59 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:31.711580 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f638770d-2fd4-4d25-be2a-f3edab304b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:31.725294 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-5cb67122-95a5-4505-8792-84e270902c5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31cd0563b999440fbe93f7d4bce54eca', 'request_id': 'req-5cb67122-95a5-4505-8792-84e270902c5d'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:31.731475 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:31.734007 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-5cb67122-95a5-4505-8792-84e270902c5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31cd0563b999440fbe93f7d4bce54eca', 'request_id': 'req-5cb67122-95a5-4505-8792-84e270902c5d'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:31.754408 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-5cb67122-95a5-4505-8792-84e270902c5d admin admin] The token cache is being invalidate because project 31cd0563b999440fbe93f7d4bce54eca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:07:31.754705 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-5cb67122-95a5-4505-8792-84e270902c5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:31.755374 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-5cb67122-95a5-4505-8792-84e270902c5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:31.756180 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f638770d-2fd4-4d25-be2a-f3edab304b59 None None] MFA Rules not processed for user `74e7b6f4cd47468e9e5ea62befe4abf4`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:31.760040 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1017/2034] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:07:31 2022] DELETE /identity/v3/projects/31cd0563b999440fbe93f7d4bce54eca => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:31.760325 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f638770d-2fd4-4d25-be2a-f3edab304b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:31.761430 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb67122-95a5-4505-8792-84e270902c5d admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:31.762835 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5cb67122-95a5-4505-8792-84e270902c5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:31.836514 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1018/2035] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:31.837963 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb67122-95a5-4505-8792-84e270902c5d admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:31.840069 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f638770d-2fd4-4d25-be2a-f3edab304b59 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:31.842622 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f638770d-2fd4-4d25-be2a-f3edab304b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:31.929567 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:31.932769 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170819aa-4d03-4dfc-84a4-f199793ba385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:31.933426 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170819aa-4d03-4dfc-84a4-f199793ba385 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:31.933709 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170819aa-4d03-4dfc-84a4-f199793ba385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:31.934066 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170819aa-4d03-4dfc-84a4-f199793ba385 admin admin] PATH_INFO: `/v3/users/707e05437ff142fd836d11ad9b3fbe8f` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:31.935955 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170819aa-4d03-4dfc-84a4-f199793ba385 admin admin] RBAC: Authorizing `identity:delete_user(user_id=707e05437ff142fd836d11ad9b3fbe8f)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:31.938592 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170819aa-4d03-4dfc-84a4-f199793ba385 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:31.944799 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f638770d-2fd4-4d25-be2a-f3edab304b59 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:31.951375 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568db75e-b240-4d1a-9abf-eab1d30dedf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:31.952018 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568db75e-b240-4d1a-9abf-eab1d30dedf7 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:31.952689 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568db75e-b240-4d1a-9abf-eab1d30dedf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:31.952689 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568db75e-b240-4d1a-9abf-eab1d30dedf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:31.953284 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568db75e-b240-4d1a-9abf-eab1d30dedf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:31.956540 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568db75e-b240-4d1a-9abf-eab1d30dedf7 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:32.092157 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1019/2036] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:07:31 2022] POST /identity/v3/projects => generated 383 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:32.096362 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568db75e-b240-4d1a-9abf-eab1d30dedf7 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:32.145242 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:32.159407 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-170819aa-4d03-4dfc-84a4-f199793ba385 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '707e05437ff142fd836d11ad9b3fbe8f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:32.160183 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-170819aa-4d03-4dfc-84a4-f199793ba385 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '707e05437ff142fd836d11ad9b3fbe8f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:32.161112 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568db75e-b240-4d1a-9abf-eab1d30dedf7 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:32.165910 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb522e-ef68-47e0-8ff3-8050293108a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:32.166521 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb522e-ef68-47e0-8ff3-8050293108a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:32.166806 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb522e-ef68-47e0-8ff3-8050293108a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:32.167086 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb522e-ef68-47e0-8ff3-8050293108a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:32.167722 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cb522e-ef68-47e0-8ff3-8050293108a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:32.169714 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cb522e-ef68-47e0-8ff3-8050293108a3 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:32.225324 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-170819aa-4d03-4dfc-84a4-f199793ba385 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '707e05437ff142fd836d11ad9b3fbe8f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:32.226057 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-170819aa-4d03-4dfc-84a4-f199793ba385 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '707e05437ff142fd836d11ad9b3fbe8f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:32.239725 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-170819aa-4d03-4dfc-84a4-f199793ba385 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '707e05437ff142fd836d11ad9b3fbe8f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:32.248850 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1020/2037] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:07:32 2022] POST /identity/v3/users => generated 298 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:07:32.249571 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb522e-ef68-47e0-8ff3-8050293108a3 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:32.252074 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1018/2038] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:07:31 2022] DELETE /identity/v3/users/707e05437ff142fd836d11ad9b3fbe8f => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:32.253288 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-51cb522e-ef68-47e0-8ff3-8050293108a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:32.253694 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170819aa-4d03-4dfc-84a4-f199793ba385 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:32.255070 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-170819aa-4d03-4dfc-84a4-f199793ba385 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:32.265881 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb522e-ef68-47e0-8ff3-8050293108a3 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:32.328187 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170819aa-4d03-4dfc-84a4-f199793ba385 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:32.332541 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5817b2f5-697a-4278-a736-b9a2b6d79b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:32.333175 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817b2f5-697a-4278-a736-b9a2b6d79b4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:32.333457 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817b2f5-697a-4278-a736-b9a2b6d79b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:32.333582 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db485109-f3d2-4b88-854b-9785e95305cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:32.333736 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817b2f5-697a-4278-a736-b9a2b6d79b4b admin admin] PATH_INFO: `/v3/projects/fb85b90af9f44e3795934f331f7b1bcd` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:32.334268 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db485109-f3d2-4b88-854b-9785e95305cb admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:32.334480 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db485109-f3d2-4b88-854b-9785e95305cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:32.334820 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db485109-f3d2-4b88-854b-9785e95305cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:32.335021 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5817b2f5-697a-4278-a736-b9a2b6d79b4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb85b90af9f44e3795934f331f7b1bcd)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:32.335474 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db485109-f3d2-4b88-854b-9785e95305cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:32.336895 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5817b2f5-697a-4278-a736-b9a2b6d79b4b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:32.337311 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db485109-f3d2-4b88-854b-9785e95305cb admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:32.350658 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1019/2039] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:32.353365 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db485109-f3d2-4b88-854b-9785e95305cb admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:32.364278 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db485109-f3d2-4b88-854b-9785e95305cb admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:32.369174 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a51441-73d7-48d1-b302-c7da3e34de89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:32.369781 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a51441-73d7-48d1-b302-c7da3e34de89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:32.370067 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a51441-73d7-48d1-b302-c7da3e34de89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:32.370347 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a51441-73d7-48d1-b302-c7da3e34de89 admin admin] PATH_INFO: `/v3/projects/587d827123c44842b575b43d967d8b68/users/7d8b4b0cf5c24129b6683051682aed55/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:32.384974 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a51441-73d7-48d1-b302-c7da3e34de89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=587d827123c44842b575b43d967d8b68, user_id=7d8b4b0cf5c24129b6683051682aed55, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:32.388550 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a51441-73d7-48d1-b302-c7da3e34de89 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:32.529551 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1020/2040] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:32 2022] PUT /identity/v3/projects/587d827123c44842b575b43d967d8b68/users/7d8b4b0cf5c24129b6683051682aed55/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:32.534077 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a51441-73d7-48d1-b302-c7da3e34de89 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:32.554331 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5817b2f5-697a-4278-a736-b9a2b6d79b4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb85b90af9f44e3795934f331f7b1bcd', 'request_id': 'req-5817b2f5-697a-4278-a736-b9a2b6d79b4b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:32.635679 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a51441-73d7-48d1-b302-c7da3e34de89 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:32.650924 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5817b2f5-697a-4278-a736-b9a2b6d79b4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb85b90af9f44e3795934f331f7b1bcd', 'request_id': 'req-5817b2f5-697a-4278-a736-b9a2b6d79b4b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:32.660330 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5817b2f5-697a-4278-a736-b9a2b6d79b4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb85b90af9f44e3795934f331f7b1bcd', 'request_id': 'req-5817b2f5-697a-4278-a736-b9a2b6d79b4b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:32.679263 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5817b2f5-697a-4278-a736-b9a2b6d79b4b admin admin] The token cache is being invalidate because project fb85b90af9f44e3795934f331f7b1bcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:07:32.679563 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5817b2f5-697a-4278-a736-b9a2b6d79b4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:32.680231 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5817b2f5-697a-4278-a736-b9a2b6d79b4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:32.681891 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1021/2041] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:07:32 2022] DELETE /identity/v3/projects/fb85b90af9f44e3795934f331f7b1bcd => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:32.685742 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5817b2f5-697a-4278-a736-b9a2b6d79b4b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:32.687642 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5817b2f5-697a-4278-a736-b9a2b6d79b4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:32.717545 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f37f7-ddbe-4834-9064-2c52c04f3495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:32.718158 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f37f7-ddbe-4834-9064-2c52c04f3495 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:32.718684 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f37f7-ddbe-4834-9064-2c52c04f3495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:32.718968 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f37f7-ddbe-4834-9064-2c52c04f3495 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:32.719616 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f37f7-ddbe-4834-9064-2c52c04f3495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:32.721805 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f37f7-ddbe-4834-9064-2c52c04f3495 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:32.733003 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1021/2042] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:32.737364 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f37f7-ddbe-4834-9064-2c52c04f3495 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:32.739447 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-552f37f7-ddbe-4834-9064-2c52c04f3495 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:32.753677 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f37f7-ddbe-4834-9064-2c52c04f3495 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:32.759947 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc20031-8ab7-4835-a0ff-a3c260a40f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:32.760658 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc20031-8ab7-4835-a0ff-a3c260a40f4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:32.760943 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc20031-8ab7-4835-a0ff-a3c260a40f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:32.761231 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc20031-8ab7-4835-a0ff-a3c260a40f4b admin admin] PATH_INFO: `/v3/projects/587d827123c44842b575b43d967d8b68/users/7d8b4b0cf5c24129b6683051682aed55/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:32.763519 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc20031-8ab7-4835-a0ff-a3c260a40f4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=587d827123c44842b575b43d967d8b68, user_id=7d8b4b0cf5c24129b6683051682aed55, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:32.763626 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5817b2f5-697a-4278-a736-b9a2b6d79b4b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:32.766255 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc20031-8ab7-4835-a0ff-a3c260a40f4b admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:32.769555 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b0e15-60b0-4e39-b4e5-3e7192460a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:32.770224 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b0e15-60b0-4e39-b4e5-3e7192460a91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:32.771257 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b0e15-60b0-4e39-b4e5-3e7192460a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:32.771257 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b0e15-60b0-4e39-b4e5-3e7192460a91 admin admin] PATH_INFO: `/v3/users/deb8f9e8938841d787442e8e46e62195` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:32.772411 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b0e15-60b0-4e39-b4e5-3e7192460a91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=deb8f9e8938841d787442e8e46e62195)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:32.774573 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b0e15-60b0-4e39-b4e5-3e7192460a91 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:32.780717 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1022/2043] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:32 2022] PUT /identity/v3/projects/587d827123c44842b575b43d967d8b68/users/7d8b4b0cf5c24129b6683051682aed55/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:32.787272 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc20031-8ab7-4835-a0ff-a3c260a40f4b admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:32.790262 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-0bc20031-8ab7-4835-a0ff-a3c260a40f4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:32.874348 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc20031-8ab7-4835-a0ff-a3c260a40f4b admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:32.880271 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61573063-e680-4b4a-a155-2e37404a642b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:32.880996 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61573063-e680-4b4a-a155-2e37404a642b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:32.881279 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61573063-e680-4b4a-a155-2e37404a642b admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:32.881555 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61573063-e680-4b4a-a155-2e37404a642b admin admin] PATH_INFO: `/v3/projects/ab06f0b03ab440b7bfe73e8a7d7b606b` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:32.882681 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61573063-e680-4b4a-a155-2e37404a642b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab06f0b03ab440b7bfe73e8a7d7b606b)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:32.884630 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61573063-e680-4b4a-a155-2e37404a642b admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:32.907708 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:33.001617 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8d4b0e15-60b0-4e39-b4e5-3e7192460a91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'deb8f9e8938841d787442e8e46e62195'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:33.003036 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8d4b0e15-60b0-4e39-b4e5-3e7192460a91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'deb8f9e8938841d787442e8e46e62195'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:33.013893 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8d4b0e15-60b0-4e39-b4e5-3e7192460a91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'deb8f9e8938841d787442e8e46e62195'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:33.035329 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-61573063-e680-4b4a-a155-2e37404a642b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab06f0b03ab440b7bfe73e8a7d7b606b', 'request_id': 'req-61573063-e680-4b4a-a155-2e37404a642b'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:33.105777 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8d4b0e15-60b0-4e39-b4e5-3e7192460a91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'deb8f9e8938841d787442e8e46e62195'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:33.108907 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8d4b0e15-60b0-4e39-b4e5-3e7192460a91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'deb8f9e8938841d787442e8e46e62195'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:33.132558 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1022/2044] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:07:32 2022] DELETE /identity/v3/users/deb8f9e8938841d787442e8e46e62195 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:33.136685 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d888d-dff3-47ef-9761-f88980be8908 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:33.136975 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d888d-dff3-47ef-9761-f88980be8908 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:33.137251 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d888d-dff3-47ef-9761-f88980be8908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:33.164617 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-61573063-e680-4b4a-a155-2e37404a642b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab06f0b03ab440b7bfe73e8a7d7b606b', 'request_id': 'req-61573063-e680-4b4a-a155-2e37404a642b'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:33.171837 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c18d888d-dff3-47ef-9761-f88980be8908 None None] MFA Rules not processed for user `7d8b4b0cf5c24129b6683051682aed55`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:33.172019 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-61573063-e680-4b4a-a155-2e37404a642b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab06f0b03ab440b7bfe73e8a7d7b606b', 'request_id': 'req-61573063-e680-4b4a-a155-2e37404a642b'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:33.173721 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c18d888d-dff3-47ef-9761-f88980be8908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:33.187938 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-61573063-e680-4b4a-a155-2e37404a642b admin admin] The token cache is being invalidate because project ab06f0b03ab440b7bfe73e8a7d7b606b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:07:33.188328 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-61573063-e680-4b4a-a155-2e37404a642b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:33.191444 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-61573063-e680-4b4a-a155-2e37404a642b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:33.198588 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1023/2045] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:07:32 2022] DELETE /identity/v3/projects/ab06f0b03ab440b7bfe73e8a7d7b606b => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:33.200726 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9386185-0c8b-4cba-a8c2-1bb5e76294e4 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:33.201084 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9386185-0c8b-4cba-a8c2-1bb5e76294e4 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:33.201443 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9386185-0c8b-4cba-a8c2-1bb5e76294e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:33.244675 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-f9386185-0c8b-4cba-a8c2-1bb5e76294e4 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:33.246024 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f9386185-0c8b-4cba-a8c2-1bb5e76294e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:33.267667 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1023/2046] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:33 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:33.278701 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d888d-dff3-47ef-9761-f88980be8908 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:33.280112 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c18d888d-dff3-47ef-9761-f88980be8908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:33.310274 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1024/2047] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:33.318248 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa339e8-15d7-4458-b61b-65bd6f652b62 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:33.318536 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa339e8-15d7-4458-b61b-65bd6f652b62 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:33.318806 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa339e8-15d7-4458-b61b-65bd6f652b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:33.340338 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-efa339e8-15d7-4458-b61b-65bd6f652b62 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:33.342289 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-efa339e8-15d7-4458-b61b-65bd6f652b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:33.342389 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d888d-dff3-47ef-9761-f88980be8908 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:33.346573 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fbeaf-0ec1-46ff-bfc9-45679ff7435b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:33.347391 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fbeaf-0ec1-46ff-bfc9-45679ff7435b service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:33.347679 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fbeaf-0ec1-46ff-bfc9-45679ff7435b service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:33.348092 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fbeaf-0ec1-46ff-bfc9-45679ff7435b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:33.349980 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-693fbeaf-0ec1-46ff-bfc9-45679ff7435b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:33.351769 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1025/2048] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:07:33.356138 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa339e8-15d7-4458-b61b-65bd6f652b62 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:33.370072 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa339e8-15d7-4458-b61b-65bd6f652b62 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:33.375576 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2596e93-935c-459a-9fc3-67ad3d6b9dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:33.376791 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2596e93-935c-459a-9fc3-67ad3d6b9dbd admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:33.377078 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2596e93-935c-459a-9fc3-67ad3d6b9dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:33.377442 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2596e93-935c-459a-9fc3-67ad3d6b9dbd admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:33.378195 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2596e93-935c-459a-9fc3-67ad3d6b9dbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:07:33.378472 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2596e93-935c-459a-9fc3-67ad3d6b9dbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:33.380711 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2596e93-935c-459a-9fc3-67ad3d6b9dbd admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:33.395613 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1026/2049] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:07:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 01:07:33.400058 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2596e93-935c-459a-9fc3-67ad3d6b9dbd admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:33.411849 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2596e93-935c-459a-9fc3-67ad3d6b9dbd admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:33.417051 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb0a2d2-84e6-4792-a972-3cae9cb5e59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:33.417662 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0a2d2-84e6-4792-a972-3cae9cb5e59a admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:33.417948 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0a2d2-84e6-4792-a972-3cae9cb5e59a admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:33.418224 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0a2d2-84e6-4792-a972-3cae9cb5e59a admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:33.418936 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb0a2d2-84e6-4792-a972-3cae9cb5e59a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:33.420943 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb0a2d2-84e6-4792-a972-3cae9cb5e59a admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:33.427194 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693fbeaf-0ec1-46ff-bfc9-45679ff7435b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:33.428686 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693fbeaf-0ec1-46ff-bfc9-45679ff7435b service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:33.440607 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1024/2050] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 01:07:33.510238 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1027/2051] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:07:33 2022] POST /identity/v3/projects => generated 397 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:07:33.514745 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fbeaf-0ec1-46ff-bfc9-45679ff7435b service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:33.527229 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fbeaf-0ec1-46ff-bfc9-45679ff7435b service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:33.584632 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b35e3-6997-440d-bd39-b4445f38e4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:33.585242 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b35e3-6997-440d-bd39-b4445f38e4ab admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:33.585520 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b35e3-6997-440d-bd39-b4445f38e4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:33.585788 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b35e3-6997-440d-bd39-b4445f38e4ab admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:33.586433 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9b35e3-6997-440d-bd39-b4445f38e4ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:33.588636 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9b35e3-6997-440d-bd39-b4445f38e4ab admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:33.589976 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb0a2d2-84e6-4792-a972-3cae9cb5e59a admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:33.592145 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-7bb0a2d2-84e6-4792-a972-3cae9cb5e59a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:33.606982 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb0a2d2-84e6-4792-a972-3cae9cb5e59a admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:33.612621 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c05bcb-8a8e-4758-af02-e2ede59b86fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:33.613269 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c05bcb-8a8e-4758-af02-e2ede59b86fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:33.613595 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c05bcb-8a8e-4758-af02-e2ede59b86fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:33.613824 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c05bcb-8a8e-4758-af02-e2ede59b86fd admin admin] PATH_INFO: `/v3/projects/1d143b484b354585b57992eb8f96576e` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:33.615022 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c05bcb-8a8e-4758-af02-e2ede59b86fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d143b484b354585b57992eb8f96576e)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:33.617169 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c05bcb-8a8e-4758-af02-e2ede59b86fd admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:33.673479 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1025/2052] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:07:33 2022] POST /identity/v3/users => generated 299 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:07:33.681128 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b35e3-6997-440d-bd39-b4445f38e4ab admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:33.691654 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b35e3-6997-440d-bd39-b4445f38e4ab admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:33.697653 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9447f79-2f3b-441f-8f9c-b6cc58aeefa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:33.698250 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9447f79-2f3b-441f-8f9c-b6cc58aeefa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:33.698524 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9447f79-2f3b-441f-8f9c-b6cc58aeefa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:33.698798 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9447f79-2f3b-441f-8f9c-b6cc58aeefa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:33.699430 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9447f79-2f3b-441f-8f9c-b6cc58aeefa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:33.701132 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9447f79-2f3b-441f-8f9c-b6cc58aeefa1 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:33.712198 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1026/2053] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 01:07:33.716380 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9447f79-2f3b-441f-8f9c-b6cc58aeefa1 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:33.726796 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9447f79-2f3b-441f-8f9c-b6cc58aeefa1 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:33.731485 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526d647-0077-4b3a-8544-7c5e123ade7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:33.732066 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526d647-0077-4b3a-8544-7c5e123ade7d admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:33.732508 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526d647-0077-4b3a-8544-7c5e123ade7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:33.732872 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526d647-0077-4b3a-8544-7c5e123ade7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:33.733532 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c526d647-0077-4b3a-8544-7c5e123ade7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:33.735804 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c526d647-0077-4b3a-8544-7c5e123ade7d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:33.747319 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1027/2054] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 01:07:33.756120 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526d647-0077-4b3a-8544-7c5e123ade7d admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:33.764373 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b8c05bcb-8a8e-4758-af02-e2ede59b86fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d143b484b354585b57992eb8f96576e', 'request_id': 'req-b8c05bcb-8a8e-4758-af02-e2ede59b86fd'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:33.767282 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526d647-0077-4b3a-8544-7c5e123ade7d admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:33.822275 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:33.827915 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8cc61-593f-44ed-9011-62652502ad5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:33.828710 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8cc61-593f-44ed-9011-62652502ad5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:33.828994 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8cc61-593f-44ed-9011-62652502ad5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:33.829246 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8cc61-593f-44ed-9011-62652502ad5b admin admin] PATH_INFO: `/v3/projects/2999291ee17b4c08a7a765b7c52588cb/users/6d853d2ebd3c406282285feef458c200/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:33.845198 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f8cc61-593f-44ed-9011-62652502ad5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2999291ee17b4c08a7a765b7c52588cb, user_id=6d853d2ebd3c406282285feef458c200, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:33.847866 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f8cc61-593f-44ed-9011-62652502ad5b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:33.850818 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b8c05bcb-8a8e-4758-af02-e2ede59b86fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d143b484b354585b57992eb8f96576e', 'request_id': 'req-b8c05bcb-8a8e-4758-af02-e2ede59b86fd'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:33.859944 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b8c05bcb-8a8e-4758-af02-e2ede59b86fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d143b484b354585b57992eb8f96576e', 'request_id': 'req-b8c05bcb-8a8e-4758-af02-e2ede59b86fd'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:33.874919 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b8c05bcb-8a8e-4758-af02-e2ede59b86fd admin admin] The token cache is being invalidate because project 1d143b484b354585b57992eb8f96576e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:07:33.875301 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b8c05bcb-8a8e-4758-af02-e2ede59b86fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:33.876053 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b8c05bcb-8a8e-4758-af02-e2ede59b86fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:33.879516 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1028/2055] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:07:33 2022] DELETE /identity/v3/projects/1d143b484b354585b57992eb8f96576e => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:07:33.883990 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65c56d-6224-400a-8d99-47ec8a453ccd None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:33.884263 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65c56d-6224-400a-8d99-47ec8a453ccd None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:33.884583 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65c56d-6224-400a-8d99-47ec8a453ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:33.905197 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-dd65c56d-6224-400a-8d99-47ec8a453ccd None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:33.906521 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-dd65c56d-6224-400a-8d99-47ec8a453ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:33.917620 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1029/2056] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:07:33.926051 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb657c-0485-49f7-95c4-151683f1f827 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:33.926361 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb657c-0485-49f7-95c4-151683f1f827 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:33.926616 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb657c-0485-49f7-95c4-151683f1f827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:33.967837 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-b1eb657c-0485-49f7-95c4-151683f1f827 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:33.969210 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b1eb657c-0485-49f7-95c4-151683f1f827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:33.971202 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1028/2057] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:33 2022] PUT /identity/v3/projects/2999291ee17b4c08a7a765b7c52588cb/users/6d853d2ebd3c406282285feef458c200/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:33.972155 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8cc61-593f-44ed-9011-62652502ad5b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:33.981920 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c4f8cc61-593f-44ed-9011-62652502ad5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:34.034684 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1030/2058] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:34.036208 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93772849-60dc-4e86-a247-f49fd6b9b05d None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:34.036601 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93772849-60dc-4e86-a247-f49fd6b9b05d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:34.036929 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93772849-60dc-4e86-a247-f49fd6b9b05d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:34.049850 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8cc61-593f-44ed-9011-62652502ad5b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:34.053605 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23403ba1-225e-444a-9199-ebbec61ed403 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:34.054204 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23403ba1-225e-444a-9199-ebbec61ed403 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:34.054485 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23403ba1-225e-444a-9199-ebbec61ed403 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:34.054762 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23403ba1-225e-444a-9199-ebbec61ed403 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:34.056545 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-23403ba1-225e-444a-9199-ebbec61ed403 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:34.069747 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-93772849-60dc-4e86-a247-f49fd6b9b05d None None] MFA Rules not processed for user `6d853d2ebd3c406282285feef458c200`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:34.071153 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-93772849-60dc-4e86-a247-f49fd6b9b05d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:34.124321 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23403ba1-225e-444a-9199-ebbec61ed403 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:34.126363 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23403ba1-225e-444a-9199-ebbec61ed403 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:34.136876 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1029/2059] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:33 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:07:34.137801 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23403ba1-225e-444a-9199-ebbec61ed403 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:34.154688 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23403ba1-225e-444a-9199-ebbec61ed403 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:34.164325 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1031/2060] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:34 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:34.176291 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b55d3-004a-43f0-a03a-9fde179a7d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:34.177083 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b55d3-004a-43f0-a03a-9fde179a7d65 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:34.177421 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b55d3-004a-43f0-a03a-9fde179a7d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:34.179403 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b55d3-004a-43f0-a03a-9fde179a7d65 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:34.179403 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b55d3-004a-43f0-a03a-9fde179a7d65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:07:34.179403 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b55d3-004a-43f0-a03a-9fde179a7d65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:34.180761 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b55d3-004a-43f0-a03a-9fde179a7d65 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:34.191934 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93772849-60dc-4e86-a247-f49fd6b9b05d None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:34.193403 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-93772849-60dc-4e86-a247-f49fd6b9b05d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:34.199158 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1030/2061] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:07:34.200975 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b55d3-004a-43f0-a03a-9fde179a7d65 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:34.213617 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b55d3-004a-43f0-a03a-9fde179a7d65 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:34.216742 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93772849-60dc-4e86-a247-f49fd6b9b05d None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:34.224173 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed6702d-cdb0-499a-9c40-ef48259eabc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:34.225353 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6702d-cdb0-499a-9c40-ef48259eabc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:34.226366 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b85872-b834-4984-b805-b1e7446832da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:34.227178 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b85872-b834-4984-b805-b1e7446832da service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:34.228231 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6702d-cdb0-499a-9c40-ef48259eabc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:34.228549 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6702d-cdb0-499a-9c40-ef48259eabc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:34.229221 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed6702d-cdb0-499a-9c40-ef48259eabc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:34.232057 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed6702d-cdb0-499a-9c40-ef48259eabc3 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:34.236894 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b85872-b834-4984-b805-b1e7446832da service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:34.237446 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b85872-b834-4984-b805-b1e7446832da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:34.252762 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b85872-b834-4984-b805-b1e7446832da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:34.254232 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b85872-b834-4984-b805-b1e7446832da service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:34.264178 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1032/2062] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:34 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:07:34.383735 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1031/2063] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:07:34 2022] POST /identity/v3/projects => generated 377 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:34.391996 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b85872-b834-4984-b805-b1e7446832da service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:34.404699 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b85872-b834-4984-b805-b1e7446832da service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:34.477129 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6ba69-8915-43ab-b047-f60035f985a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:34.480983 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6ba69-8915-43ab-b047-f60035f985a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:34.480983 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6ba69-8915-43ab-b047-f60035f985a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:34.480983 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6ba69-8915-43ab-b047-f60035f985a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:34.480983 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f6ba69-8915-43ab-b047-f60035f985a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:34.481315 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f6ba69-8915-43ab-b047-f60035f985a6 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:34.572300 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1033/2064] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:07:34 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:07:34.576309 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed6702d-cdb0-499a-9c40-ef48259eabc3 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:34.588313 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed6702d-cdb0-499a-9c40-ef48259eabc3 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:34.593841 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:34.597987 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68881086-f078-4ebb-aeb6-c7ea67ba1b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:34.598606 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68881086-f078-4ebb-aeb6-c7ea67ba1b74 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:34.598898 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68881086-f078-4ebb-aeb6-c7ea67ba1b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:34.599216 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68881086-f078-4ebb-aeb6-c7ea67ba1b74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:34.599894 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68881086-f078-4ebb-aeb6-c7ea67ba1b74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:34.601552 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68881086-f078-4ebb-aeb6-c7ea67ba1b74 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:34.613834 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1032/2065] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:34.618418 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6ba69-8915-43ab-b047-f60035f985a6 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:34.629434 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6ba69-8915-43ab-b047-f60035f985a6 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:34.644963 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e24d39-95fb-42db-b5e4-7a084f4afa5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:34.649145 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e24d39-95fb-42db-b5e4-7a084f4afa5e admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:34.649736 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e24d39-95fb-42db-b5e4-7a084f4afa5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:34.650161 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e24d39-95fb-42db-b5e4-7a084f4afa5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:34.651919 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e24d39-95fb-42db-b5e4-7a084f4afa5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:34.653826 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e24d39-95fb-42db-b5e4-7a084f4afa5e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:34.672100 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1034/2066] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:34.673798 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68881086-f078-4ebb-aeb6-c7ea67ba1b74 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:34.685356 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68881086-f078-4ebb-aeb6-c7ea67ba1b74 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:34.691302 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425c0ef-8972-4570-9cf7-901f8d913bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:34.692021 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425c0ef-8972-4570-9cf7-901f8d913bac admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:34.692301 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425c0ef-8972-4570-9cf7-901f8d913bac admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:34.692612 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425c0ef-8972-4570-9cf7-901f8d913bac admin admin] PATH_INFO: `/v3/projects/f6e9daaec12642908434416f4c275b29/users/f2acc3fe07a24687bc768d86e21d8de1/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:34.722347 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3425c0ef-8972-4570-9cf7-901f8d913bac admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6e9daaec12642908434416f4c275b29, user_id=f2acc3fe07a24687bc768d86e21d8de1, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:34.725789 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3425c0ef-8972-4570-9cf7-901f8d913bac admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:34.844090 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1033/2067] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:34 2022] PUT /identity/v3/projects/f6e9daaec12642908434416f4c275b29/users/f2acc3fe07a24687bc768d86e21d8de1/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:34.855956 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8ea19-9c46-49df-820c-cf2bab4d969f None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:34.856241 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8ea19-9c46-49df-820c-cf2bab4d969f None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:34.856611 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8ea19-9c46-49df-820c-cf2bab4d969f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:34.885942 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-6cf8ea19-9c46-49df-820c-cf2bab4d969f None None] MFA Rules not processed for user `f2acc3fe07a24687bc768d86e21d8de1`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:34.887210 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-6cf8ea19-9c46-49df-820c-cf2bab4d969f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:34.963918 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1035/2068] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:34.968435 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425c0ef-8972-4570-9cf7-901f8d913bac admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:35.035139 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425c0ef-8972-4570-9cf7-901f8d913bac admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:35.039854 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451938b-dc40-4fdb-b9a9-6c5ae1faa864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:35.040502 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451938b-dc40-4fdb-b9a9-6c5ae1faa864 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:35.040775 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451938b-dc40-4fdb-b9a9-6c5ae1faa864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:35.041053 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451938b-dc40-4fdb-b9a9-6c5ae1faa864 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:35.041705 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e451938b-dc40-4fdb-b9a9-6c5ae1faa864 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:35.043803 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e451938b-dc40-4fdb-b9a9-6c5ae1faa864 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:35.187381 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1034/2069] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:07:34 2022] POST /identity/v3/projects => generated 375 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:35.192605 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf8ea19-9c46-49df-820c-cf2bab4d969f None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:35.217471 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf8ea19-9c46-49df-820c-cf2bab4d969f None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:35.296245 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23790050-4812-4248-b7f8-5740ed41d7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:35.297020 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23790050-4812-4248-b7f8-5740ed41d7fb admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:35.297503 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23790050-4812-4248-b7f8-5740ed41d7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:35.297950 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23790050-4812-4248-b7f8-5740ed41d7fb admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:35.298850 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23790050-4812-4248-b7f8-5740ed41d7fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:35.302305 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23790050-4812-4248-b7f8-5740ed41d7fb admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:35.394949 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1036/2070] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:07:35 2022] POST /identity/v3/users => generated 294 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:35.398103 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451938b-dc40-4fdb-b9a9-6c5ae1faa864 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:35.409293 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451938b-dc40-4fdb-b9a9-6c5ae1faa864 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:35.414689 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad766eb-a1f6-4bf5-bea3-e7961b35333c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:35.415656 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad766eb-a1f6-4bf5-bea3-e7961b35333c admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:35.415772 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad766eb-a1f6-4bf5-bea3-e7961b35333c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:35.416074 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad766eb-a1f6-4bf5-bea3-e7961b35333c admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:35.416855 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad766eb-a1f6-4bf5-bea3-e7961b35333c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:35.418598 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad766eb-a1f6-4bf5-bea3-e7961b35333c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:35.432107 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1035/2071] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:35.436182 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23790050-4812-4248-b7f8-5740ed41d7fb admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:35.447283 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23790050-4812-4248-b7f8-5740ed41d7fb admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:35.452298 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3decc9-d503-4ef4-b839-df91e78aa746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:35.452997 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3decc9-d503-4ef4-b839-df91e78aa746 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:35.453277 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3decc9-d503-4ef4-b839-df91e78aa746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:35.453550 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3decc9-d503-4ef4-b839-df91e78aa746 admin admin] PATH_INFO: `/v3/projects/4aa21864731646f0b73ac2d877c9aadc/users/55f2f29844374152bc1ebd338074cd18/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:35.470013 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3decc9-d503-4ef4-b839-df91e78aa746 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aa21864731646f0b73ac2d877c9aadc, user_id=55f2f29844374152bc1ebd338074cd18, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:35.473404 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3decc9-d503-4ef4-b839-df91e78aa746 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:35.583061 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1037/2072] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:35 2022] PUT /identity/v3/projects/4aa21864731646f0b73ac2d877c9aadc/users/55f2f29844374152bc1ebd338074cd18/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:35.584922 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad766eb-a1f6-4bf5-bea3-e7961b35333c admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:35.595109 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad766eb-a1f6-4bf5-bea3-e7961b35333c admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:35.693509 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b6760-4882-422e-b61c-763e9ede718e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:35.694690 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b6760-4882-422e-b61c-763e9ede718e admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:35.694983 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b6760-4882-422e-b61c-763e9ede718e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:35.695262 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b6760-4882-422e-b61c-763e9ede718e admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:35.695913 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b6760-4882-422e-b61c-763e9ede718e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:35.697945 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b6760-4882-422e-b61c-763e9ede718e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:35.720000 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1036/2073] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:35.721369 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3decc9-d503-4ef4-b839-df91e78aa746 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:35.744776 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3decc9-d503-4ef4-b839-df91e78aa746 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:35.746793 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd52dd1-a684-4684-9beb-6bf2570bc03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:35.747915 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd52dd1-a684-4684-9beb-6bf2570bc03a admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:35.747915 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd52dd1-a684-4684-9beb-6bf2570bc03a admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:35.748162 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd52dd1-a684-4684-9beb-6bf2570bc03a admin admin] PATH_INFO: `/v3/projects/4aa21864731646f0b73ac2d877c9aadc/users/55f2f29844374152bc1ebd338074cd18/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:35.750271 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd52dd1-a684-4684-9beb-6bf2570bc03a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aa21864731646f0b73ac2d877c9aadc, user_id=55f2f29844374152bc1ebd338074cd18, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:35.754206 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd52dd1-a684-4684-9beb-6bf2570bc03a admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:35.772499 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1038/2074] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:35 2022] PUT /identity/v3/projects/4aa21864731646f0b73ac2d877c9aadc/users/55f2f29844374152bc1ebd338074cd18/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:35.777630 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8eb97c-9770-482e-b2ab-c8f40a25b196 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:35.778348 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8eb97c-9770-482e-b2ab-c8f40a25b196 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:35.778760 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8eb97c-9770-482e-b2ab-c8f40a25b196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:35.828221 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:35.836943 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-3f8eb97c-9770-482e-b2ab-c8f40a25b196 None None] MFA Rules not processed for user `55f2f29844374152bc1ebd338074cd18`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:35.840893 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3f8eb97c-9770-482e-b2ab-c8f40a25b196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:35.973759 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1037/2075] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:35 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:35.985778 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd52dd1-a684-4684-9beb-6bf2570bc03a admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:36.046360 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd52dd1-a684-4684-9beb-6bf2570bc03a admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:36.051634 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e50179-cfb5-419c-9e38-b7cdb3a39612 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:36.052262 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e50179-cfb5-419c-9e38-b7cdb3a39612 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:36.052577 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e50179-cfb5-419c-9e38-b7cdb3a39612 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:36.053295 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e50179-cfb5-419c-9e38-b7cdb3a39612 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:36.122215 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e50179-cfb5-419c-9e38-b7cdb3a39612 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:36.123750 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e50179-cfb5-419c-9e38-b7cdb3a39612 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:36.138307 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1039/2076] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:07:37.429507 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:37.579892 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:38.341977 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:39.730942 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:39.970440 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:40.187041 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8eb97c-9770-482e-b2ab-c8f40a25b196 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:40.188661 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3f8eb97c-9770-482e-b2ab-c8f40a25b196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:40.273292 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8eb97c-9770-482e-b2ab-c8f40a25b196 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:40.278062 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b09f89-adaf-44a7-bacd-809d43bf7aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:40.278691 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b09f89-adaf-44a7-bacd-809d43bf7aec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:40.279053 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b09f89-adaf-44a7-bacd-809d43bf7aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:40.279327 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b09f89-adaf-44a7-bacd-809d43bf7aec admin admin] PATH_INFO: `/v3/users/4110e5346972427fb5a0a4cfae90ad3e` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:40.280582 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b09f89-adaf-44a7-bacd-809d43bf7aec admin admin] RBAC: Authorizing `identity:delete_user(user_id=4110e5346972427fb5a0a4cfae90ad3e)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:40.282687 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b09f89-adaf-44a7-bacd-809d43bf7aec admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:40.452371 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-02b09f89-adaf-44a7-bacd-809d43bf7aec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4110e5346972427fb5a0a4cfae90ad3e'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:40.453053 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-02b09f89-adaf-44a7-bacd-809d43bf7aec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4110e5346972427fb5a0a4cfae90ad3e'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:40.460745 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-02b09f89-adaf-44a7-bacd-809d43bf7aec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4110e5346972427fb5a0a4cfae90ad3e'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:40.573282 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-02b09f89-adaf-44a7-bacd-809d43bf7aec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4110e5346972427fb5a0a4cfae90ad3e'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:40.573904 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-02b09f89-adaf-44a7-bacd-809d43bf7aec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4110e5346972427fb5a0a4cfae90ad3e'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:40.595693 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1038/2077] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:07:40 2022] DELETE /identity/v3/users/4110e5346972427fb5a0a4cfae90ad3e => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:40.604974 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55b7c1-c62d-4ae6-960e-3e29cd982b90 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:40.604974 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55b7c1-c62d-4ae6-960e-3e29cd982b90 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:40.604974 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55b7c1-c62d-4ae6-960e-3e29cd982b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:40.621284 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-fc55b7c1-c62d-4ae6-960e-3e29cd982b90 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:40.622503 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-fc55b7c1-c62d-4ae6-960e-3e29cd982b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:40.692355 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1040/2078] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:40.698133 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b09f89-adaf-44a7-bacd-809d43bf7aec admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:40.699578 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-02b09f89-adaf-44a7-bacd-809d43bf7aec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:40.761573 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b09f89-adaf-44a7-bacd-809d43bf7aec admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:40.765511 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eafa55e-3e49-4805-af64-76f073b3f233 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:40.766107 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafa55e-3e49-4805-af64-76f073b3f233 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:40.766387 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafa55e-3e49-4805-af64-76f073b3f233 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:40.766666 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafa55e-3e49-4805-af64-76f073b3f233 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:40.780433 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eafa55e-3e49-4805-af64-76f073b3f233 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:40.781875 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eafa55e-3e49-4805-af64-76f073b3f233 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:40.792254 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1039/2079] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:07:40.950539 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:41.162509 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:41.206019 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55b7c1-c62d-4ae6-960e-3e29cd982b90 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:41.207484 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-fc55b7c1-c62d-4ae6-960e-3e29cd982b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:41.268766 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eafa55e-3e49-4805-af64-76f073b3f233 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:41.270482 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0eafa55e-3e49-4805-af64-76f073b3f233 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:41.278291 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55b7c1-c62d-4ae6-960e-3e29cd982b90 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:41.282404 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6cbf3d-6ec4-4e78-9d18-f142892804b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:41.283438 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6cbf3d-6ec4-4e78-9d18-f142892804b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:41.284266 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6cbf3d-6ec4-4e78-9d18-f142892804b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:41.284563 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6cbf3d-6ec4-4e78-9d18-f142892804b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:41.285295 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eafa55e-3e49-4805-af64-76f073b3f233 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:41.288840 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-9f6cbf3d-6ec4-4e78-9d18-f142892804b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:41.290566 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf31c89a-33da-4135-bc31-c6d2b6b1b548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:41.291148 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31c89a-33da-4135-bc31-c6d2b6b1b548 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:41.291435 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31c89a-33da-4135-bc31-c6d2b6b1b548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:41.291764 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31c89a-33da-4135-bc31-c6d2b6b1b548 admin admin] PATH_INFO: `/v3/projects/44bb8d643c484e3683a83249575277cb` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:41.292988 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf31c89a-33da-4135-bc31-c6d2b6b1b548 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44bb8d643c484e3683a83249575277cb)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:41.295062 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf31c89a-33da-4135-bc31-c6d2b6b1b548 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:41.352120 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6cbf3d-6ec4-4e78-9d18-f142892804b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:41.353641 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6cbf3d-6ec4-4e78-9d18-f142892804b8 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:41.364528 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1041/2080] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:41 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:07:41.422922 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-bf31c89a-33da-4135-bc31-c6d2b6b1b548 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44bb8d643c484e3683a83249575277cb', 'request_id': 'req-bf31c89a-33da-4135-bc31-c6d2b6b1b548'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:41.553356 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-bf31c89a-33da-4135-bc31-c6d2b6b1b548 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44bb8d643c484e3683a83249575277cb', 'request_id': 'req-bf31c89a-33da-4135-bc31-c6d2b6b1b548'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:41.560036 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-bf31c89a-33da-4135-bc31-c6d2b6b1b548 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44bb8d643c484e3683a83249575277cb', 'request_id': 'req-bf31c89a-33da-4135-bc31-c6d2b6b1b548'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:41.572187 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-bf31c89a-33da-4135-bc31-c6d2b6b1b548 admin admin] The token cache is being invalidate because project 44bb8d643c484e3683a83249575277cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:07:41.572495 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-bf31c89a-33da-4135-bc31-c6d2b6b1b548 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:41.573140 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-bf31c89a-33da-4135-bc31-c6d2b6b1b548 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:41.574597 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1040/2081] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:07:41 2022] DELETE /identity/v3/projects/44bb8d643c484e3683a83249575277cb => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:07:41.582814 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6cbf3d-6ec4-4e78-9d18-f142892804b8 service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:41.593364 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6cbf3d-6ec4-4e78-9d18-f142892804b8 service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:41.629234 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:41.651324 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2835fe-6e6b-452e-bcd8-5c217fb69813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:41.651941 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2835fe-6e6b-452e-bcd8-5c217fb69813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:41.652220 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2835fe-6e6b-452e-bcd8-5c217fb69813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:41.652518 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2835fe-6e6b-452e-bcd8-5c217fb69813 admin admin] PATH_INFO: `/v3/users/5a7cfec29d1442b4b6871eff57480187` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:41.653666 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2835fe-6e6b-452e-bcd8-5c217fb69813 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a7cfec29d1442b4b6871eff57480187)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:41.655591 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2835fe-6e6b-452e-bcd8-5c217fb69813 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:41.840856 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf31c89a-33da-4135-bc31-c6d2b6b1b548 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:41.844978 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bf31c89a-33da-4135-bc31-c6d2b6b1b548 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:41.851567 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0e2835fe-6e6b-452e-bcd8-5c217fb69813 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a7cfec29d1442b4b6871eff57480187'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:41.852328 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0e2835fe-6e6b-452e-bcd8-5c217fb69813 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a7cfec29d1442b4b6871eff57480187'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:41.900905 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf31c89a-33da-4135-bc31-c6d2b6b1b548 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:41.904794 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85eae88-a554-406b-8bf9-fb10330b8570 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:41.905420 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85eae88-a554-406b-8bf9-fb10330b8570 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:41.905701 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85eae88-a554-406b-8bf9-fb10330b8570 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:41.905973 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85eae88-a554-406b-8bf9-fb10330b8570 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:41.907746 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f85eae88-a554-406b-8bf9-fb10330b8570 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:41.927879 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0e2835fe-6e6b-452e-bcd8-5c217fb69813 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a7cfec29d1442b4b6871eff57480187'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:41.928787 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0e2835fe-6e6b-452e-bcd8-5c217fb69813 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a7cfec29d1442b4b6871eff57480187'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:41.942178 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0e2835fe-6e6b-452e-bcd8-5c217fb69813 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a7cfec29d1442b4b6871eff57480187'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:41.954073 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1042/2082] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:07:41 2022] DELETE /identity/v3/users/5a7cfec29d1442b4b6871eff57480187 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:41.972755 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85eae88-a554-406b-8bf9-fb10330b8570 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:41.974389 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85eae88-a554-406b-8bf9-fb10330b8570 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:41.984042 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1041/2083] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:41 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 01:07:42.495132 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2835fe-6e6b-452e-bcd8-5c217fb69813 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:42.495132 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-0e2835fe-6e6b-452e-bcd8-5c217fb69813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:42.514761 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:42.562620 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2835fe-6e6b-452e-bcd8-5c217fb69813 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:42.568520 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9563fd0-cc46-4fee-aa01-f2cd58c47c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:42.569168 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9563fd0-cc46-4fee-aa01-f2cd58c47c69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:42.569450 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9563fd0-cc46-4fee-aa01-f2cd58c47c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:42.569729 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9563fd0-cc46-4fee-aa01-f2cd58c47c69 admin admin] PATH_INFO: `/v3/projects/9799f42352f349bb8037b270cf3b48db` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:42.570898 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9563fd0-cc46-4fee-aa01-f2cd58c47c69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9799f42352f349bb8037b270cf3b48db)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:42.572867 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9563fd0-cc46-4fee-aa01-f2cd58c47c69 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:42.707004 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-c9563fd0-cc46-4fee-aa01-f2cd58c47c69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9799f42352f349bb8037b270cf3b48db', 'request_id': 'req-c9563fd0-cc46-4fee-aa01-f2cd58c47c69'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:42.769135 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:42.778681 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85eae88-a554-406b-8bf9-fb10330b8570 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:42.780160 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f85eae88-a554-406b-8bf9-fb10330b8570 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:42.794162 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85eae88-a554-406b-8bf9-fb10330b8570 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:42.799070 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-c9563fd0-cc46-4fee-aa01-f2cd58c47c69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9799f42352f349bb8037b270cf3b48db', 'request_id': 'req-c9563fd0-cc46-4fee-aa01-f2cd58c47c69'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:42.806108 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-c9563fd0-cc46-4fee-aa01-f2cd58c47c69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9799f42352f349bb8037b270cf3b48db', 'request_id': 'req-c9563fd0-cc46-4fee-aa01-f2cd58c47c69'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:42.818858 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-c9563fd0-cc46-4fee-aa01-f2cd58c47c69 admin admin] The token cache is being invalidate because project 9799f42352f349bb8037b270cf3b48db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:07:42.819404 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-c9563fd0-cc46-4fee-aa01-f2cd58c47c69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:42.820072 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-c9563fd0-cc46-4fee-aa01-f2cd58c47c69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:42.822444 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1043/2084] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:07:42 2022] DELETE /identity/v3/projects/9799f42352f349bb8037b270cf3b48db => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:42.826380 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9563fd0-cc46-4fee-aa01-f2cd58c47c69 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:42.829325 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c9563fd0-cc46-4fee-aa01-f2cd58c47c69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:42.855621 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f88121-3e53-4589-a0d5-4bab2e6e0895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:42.856442 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f88121-3e53-4589-a0d5-4bab2e6e0895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:42.856857 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f88121-3e53-4589-a0d5-4bab2e6e0895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:42.857141 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f88121-3e53-4589-a0d5-4bab2e6e0895 admin admin] PATH_INFO: `/v3/users/6d853d2ebd3c406282285feef458c200` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:42.861460 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f88121-3e53-4589-a0d5-4bab2e6e0895 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d853d2ebd3c406282285feef458c200)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:42.861460 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f88121-3e53-4589-a0d5-4bab2e6e0895 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:42.898803 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9563fd0-cc46-4fee-aa01-f2cd58c47c69 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:42.903661 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f29d60e-6312-4979-a694-316aa893f263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:42.904295 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29d60e-6312-4979-a694-316aa893f263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:42.904605 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29d60e-6312-4979-a694-316aa893f263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:42.907316 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29d60e-6312-4979-a694-316aa893f263 admin admin] PATH_INFO: `/v3/users/9476793c690d45019109159350a3b850` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:42.907316 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f29d60e-6312-4979-a694-316aa893f263 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9476793c690d45019109159350a3b850)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:42.908028 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f29d60e-6312-4979-a694-316aa893f263 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:43.046456 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d2f88121-3e53-4589-a0d5-4bab2e6e0895 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d853d2ebd3c406282285feef458c200'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:43.047108 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d2f88121-3e53-4589-a0d5-4bab2e6e0895 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d853d2ebd3c406282285feef458c200'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:43.054472 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d2f88121-3e53-4589-a0d5-4bab2e6e0895 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d853d2ebd3c406282285feef458c200'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:43.141441 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d2f88121-3e53-4589-a0d5-4bab2e6e0895 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d853d2ebd3c406282285feef458c200'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:43.142188 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d2f88121-3e53-4589-a0d5-4bab2e6e0895 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d853d2ebd3c406282285feef458c200'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:43.155078 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0f29d60e-6312-4979-a694-316aa893f263 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9476793c690d45019109159350a3b850'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:43.155959 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0f29d60e-6312-4979-a694-316aa893f263 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9476793c690d45019109159350a3b850'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:43.160964 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1042/2085] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:07:42 2022] DELETE /identity/v3/users/6d853d2ebd3c406282285feef458c200 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:43.165260 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807b608-98b6-4bbf-92a2-f8b9abab2084 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:43.165549 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807b608-98b6-4bbf-92a2-f8b9abab2084 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:43.165862 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807b608-98b6-4bbf-92a2-f8b9abab2084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:43.185029 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-8807b608-98b6-4bbf-92a2-f8b9abab2084 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:43.186245 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8807b608-98b6-4bbf-92a2-f8b9abab2084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:43.227138 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0f29d60e-6312-4979-a694-316aa893f263 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9476793c690d45019109159350a3b850'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:43.227894 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0f29d60e-6312-4979-a694-316aa893f263 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9476793c690d45019109159350a3b850'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:43.243053 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0f29d60e-6312-4979-a694-316aa893f263 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9476793c690d45019109159350a3b850'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:43.258062 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1044/2086] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:07:42 2022] DELETE /identity/v3/users/9476793c690d45019109159350a3b850 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:07:43.259854 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1043/2087] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:07:43.270315 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f29d60e-6312-4979-a694-316aa893f263 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:43.271797 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-0f29d60e-6312-4979-a694-316aa893f263 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:43.329813 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f29d60e-6312-4979-a694-316aa893f263 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:43.333584 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7debce4f-3002-4331-b857-f595ef05b231 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:43.334227 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7debce4f-3002-4331-b857-f595ef05b231 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:43.334509 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7debce4f-3002-4331-b857-f595ef05b231 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:43.334783 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7debce4f-3002-4331-b857-f595ef05b231 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:43.395599 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7debce4f-3002-4331-b857-f595ef05b231 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:43.397089 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7debce4f-3002-4331-b857-f595ef05b231 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:43.406809 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1045/2088] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:07:43.733814 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:43.774168 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807b608-98b6-4bbf-92a2-f8b9abab2084 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:43.775627 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8807b608-98b6-4bbf-92a2-f8b9abab2084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:43.800007 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807b608-98b6-4bbf-92a2-f8b9abab2084 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:43.806109 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e55de4-6ee5-498a-b53f-8821201fe615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:43.809030 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e55de4-6ee5-498a-b53f-8821201fe615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:43.809313 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e55de4-6ee5-498a-b53f-8821201fe615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:43.809737 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e55de4-6ee5-498a-b53f-8821201fe615 admin admin] PATH_INFO: `/v3/projects/39b32a56c3694cc1b1f5027ae1b123d2` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:43.811610 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e55de4-6ee5-498a-b53f-8821201fe615 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39b32a56c3694cc1b1f5027ae1b123d2)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:43.813831 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e55de4-6ee5-498a-b53f-8821201fe615 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:43.885145 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7debce4f-3002-4331-b857-f595ef05b231 service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:43.889018 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-7debce4f-3002-4331-b857-f595ef05b231 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:43.924721 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7debce4f-3002-4331-b857-f595ef05b231 service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:43.945509 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b8e55de4-6ee5-498a-b53f-8821201fe615 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39b32a56c3694cc1b1f5027ae1b123d2', 'request_id': 'req-b8e55de4-6ee5-498a-b53f-8821201fe615'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:44.004616 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79620d2d-fca2-4876-88e2-d6aa2c9f7670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:44.005372 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79620d2d-fca2-4876-88e2-d6aa2c9f7670 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:44.005615 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79620d2d-fca2-4876-88e2-d6aa2c9f7670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:44.005896 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79620d2d-fca2-4876-88e2-d6aa2c9f7670 admin admin] PATH_INFO: `/v3/projects/2999291ee17b4c08a7a765b7c52588cb` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:44.007054 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79620d2d-fca2-4876-88e2-d6aa2c9f7670 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2999291ee17b4c08a7a765b7c52588cb)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:44.010470 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79620d2d-fca2-4876-88e2-d6aa2c9f7670 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:44.021542 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b8e55de4-6ee5-498a-b53f-8821201fe615 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39b32a56c3694cc1b1f5027ae1b123d2', 'request_id': 'req-b8e55de4-6ee5-498a-b53f-8821201fe615'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:44.021936 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:44.030402 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b8e55de4-6ee5-498a-b53f-8821201fe615 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39b32a56c3694cc1b1f5027ae1b123d2', 'request_id': 'req-b8e55de4-6ee5-498a-b53f-8821201fe615'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:44.042927 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b8e55de4-6ee5-498a-b53f-8821201fe615 admin admin] The token cache is being invalidate because project 39b32a56c3694cc1b1f5027ae1b123d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:07:44.043223 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b8e55de4-6ee5-498a-b53f-8821201fe615 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:44.043886 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b8e55de4-6ee5-498a-b53f-8821201fe615 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:44.045577 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1044/2089] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:07:43 2022] DELETE /identity/v3/projects/39b32a56c3694cc1b1f5027ae1b123d2 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:44.052209 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2473f205-7cfe-4898-a7d0-aa3b352ed221 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:44.052209 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2473f205-7cfe-4898-a7d0-aa3b352ed221 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:44.052209 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2473f205-7cfe-4898-a7d0-aa3b352ed221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:44.073139 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-2473f205-7cfe-4898-a7d0-aa3b352ed221 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:44.074396 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2473f205-7cfe-4898-a7d0-aa3b352ed221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:44.085109 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1045/2090] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:07:44.092747 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95632133-3e2d-491c-93b7-4930de656b88 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:44.093058 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95632133-3e2d-491c-93b7-4930de656b88 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:44.093509 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95632133-3e2d-491c-93b7-4930de656b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:44.113475 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-95632133-3e2d-491c-93b7-4930de656b88 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:44.114753 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-95632133-3e2d-491c-93b7-4930de656b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:44.127106 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1046/2091] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:07:44.131976 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95632133-3e2d-491c-93b7-4930de656b88 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:44.141764 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95632133-3e2d-491c-93b7-4930de656b88 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:44.159646 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-79620d2d-fca2-4876-88e2-d6aa2c9f7670 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2999291ee17b4c08a7a765b7c52588cb', 'request_id': 'req-79620d2d-fca2-4876-88e2-d6aa2c9f7670'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:44.205351 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf8bef-fb8b-4033-9bb7-fc0bde422edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:44.205950 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf8bef-fb8b-4033-9bb7-fc0bde422edf admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:44.206236 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf8bef-fb8b-4033-9bb7-fc0bde422edf admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:44.206511 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf8bef-fb8b-4033-9bb7-fc0bde422edf admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:44.207160 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cf8bef-fb8b-4033-9bb7-fc0bde422edf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:07:44.207467 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cf8bef-fb8b-4033-9bb7-fc0bde422edf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:44.209109 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cf8bef-fb8b-4033-9bb7-fc0bde422edf admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:44.221796 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1047/2092] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 01:07:44.231424 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf8bef-fb8b-4033-9bb7-fc0bde422edf admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:44.239738 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-79620d2d-fca2-4876-88e2-d6aa2c9f7670 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2999291ee17b4c08a7a765b7c52588cb', 'request_id': 'req-79620d2d-fca2-4876-88e2-d6aa2c9f7670'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:44.245048 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf8bef-fb8b-4033-9bb7-fc0bde422edf admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:44.249482 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-79620d2d-fca2-4876-88e2-d6aa2c9f7670 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2999291ee17b4c08a7a765b7c52588cb', 'request_id': 'req-79620d2d-fca2-4876-88e2-d6aa2c9f7670'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:44.250485 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d9191-35e5-45fa-b6c2-4513dcb1ad47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:44.251239 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d9191-35e5-45fa-b6c2-4513dcb1ad47 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:44.251530 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d9191-35e5-45fa-b6c2-4513dcb1ad47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:44.251812 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d9191-35e5-45fa-b6c2-4513dcb1ad47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:44.253003 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d9191-35e5-45fa-b6c2-4513dcb1ad47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:44.255145 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d9191-35e5-45fa-b6c2-4513dcb1ad47 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:44.266783 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-79620d2d-fca2-4876-88e2-d6aa2c9f7670 admin admin] The token cache is being invalidate because project 2999291ee17b4c08a7a765b7c52588cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:07:44.267078 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-79620d2d-fca2-4876-88e2-d6aa2c9f7670 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:44.267763 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-79620d2d-fca2-4876-88e2-d6aa2c9f7670 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:44.269867 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1046/2093] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:07:43 2022] DELETE /identity/v3/projects/2999291ee17b4c08a7a765b7c52588cb => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:07:44.361090 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1048/2094] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:07:44 2022] POST /identity/v3/projects => generated 383 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:44.367931 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79620d2d-fca2-4876-88e2-d6aa2c9f7670 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:44.374404 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-79620d2d-fca2-4876-88e2-d6aa2c9f7670 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:44.426465 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79620d2d-fca2-4876-88e2-d6aa2c9f7670 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:44.521935 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e64b9-94b8-45df-b381-1c66410b6a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:44.522589 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e64b9-94b8-45df-b381-1c66410b6a96 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:44.523125 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e64b9-94b8-45df-b381-1c66410b6a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:44.523404 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e64b9-94b8-45df-b381-1c66410b6a96 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:44.524047 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437e64b9-94b8-45df-b381-1c66410b6a96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:44.526023 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437e64b9-94b8-45df-b381-1c66410b6a96 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:44.682919 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1047/2095] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:07:44 2022] POST /identity/v3/users => generated 292 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:44.687036 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d9191-35e5-45fa-b6c2-4513dcb1ad47 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:44.698260 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d9191-35e5-45fa-b6c2-4513dcb1ad47 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:44.705169 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f02e5a7-0fb6-44e6-9bb3-8569fdd30b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:44.706545 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f02e5a7-0fb6-44e6-9bb3-8569fdd30b4c admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:44.710220 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f02e5a7-0fb6-44e6-9bb3-8569fdd30b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:44.710220 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f02e5a7-0fb6-44e6-9bb3-8569fdd30b4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:44.710220 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f02e5a7-0fb6-44e6-9bb3-8569fdd30b4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:44.710220 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f02e5a7-0fb6-44e6-9bb3-8569fdd30b4c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:44.724232 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1049/2096] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:44.730142 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e64b9-94b8-45df-b381-1c66410b6a96 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:44.743886 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e64b9-94b8-45df-b381-1c66410b6a96 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:44.749192 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ade6b9-b71e-4c82-86be-bceb8641d389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:44.749877 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ade6b9-b71e-4c82-86be-bceb8641d389 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:44.750229 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ade6b9-b71e-4c82-86be-bceb8641d389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:44.750503 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ade6b9-b71e-4c82-86be-bceb8641d389 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:44.751139 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ade6b9-b71e-4c82-86be-bceb8641d389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:44.753091 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ade6b9-b71e-4c82-86be-bceb8641d389 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:44.775146 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1048/2097] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:44.779532 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f02e5a7-0fb6-44e6-9bb3-8569fdd30b4c admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:44.791093 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f02e5a7-0fb6-44e6-9bb3-8569fdd30b4c admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:44.803219 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfeafc3-3efa-41c6-880f-8d3ea52d53ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:44.803970 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfeafc3-3efa-41c6-880f-8d3ea52d53ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:44.804479 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfeafc3-3efa-41c6-880f-8d3ea52d53ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:44.804831 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfeafc3-3efa-41c6-880f-8d3ea52d53ab admin admin] PATH_INFO: `/v3/projects/924b7ec4bd73415aa64de7389fe4c004/users/2cdd53c0f23d4fb4bf7e3d68e556dff2/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:44.821133 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfeafc3-3efa-41c6-880f-8d3ea52d53ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=924b7ec4bd73415aa64de7389fe4c004, user_id=2cdd53c0f23d4fb4bf7e3d68e556dff2, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:44.823940 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfeafc3-3efa-41c6-880f-8d3ea52d53ab admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:44.911735 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1050/2098] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:44 2022] PUT /identity/v3/projects/924b7ec4bd73415aa64de7389fe4c004/users/2cdd53c0f23d4fb4bf7e3d68e556dff2/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:44.923171 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c95663-2029-4010-befa-143c99073115 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:44.923463 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c95663-2029-4010-befa-143c99073115 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:44.923740 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c95663-2029-4010-befa-143c99073115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:44.954679 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-63c95663-2029-4010-befa-143c99073115 None None] MFA Rules not processed for user `2cdd53c0f23d4fb4bf7e3d68e556dff2`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:44.955981 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-63c95663-2029-4010-befa-143c99073115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:45.021645 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1049/2099] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:07:45.026170 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfeafc3-3efa-41c6-880f-8d3ea52d53ab admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:45.091990 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfeafc3-3efa-41c6-880f-8d3ea52d53ab admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:45.098069 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94a07cd-8890-4769-b11d-5d1d91756746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:45.098687 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a07cd-8890-4769-b11d-5d1d91756746 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:45.098967 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a07cd-8890-4769-b11d-5d1d91756746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:45.099349 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a07cd-8890-4769-b11d-5d1d91756746 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:45.099912 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94a07cd-8890-4769-b11d-5d1d91756746 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:45.101946 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94a07cd-8890-4769-b11d-5d1d91756746 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:45.242764 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1051/2100] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:07:45 2022] POST /identity/v3/projects => generated 383 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:07:45.246987 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c95663-2029-4010-befa-143c99073115 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:45.257701 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c95663-2029-4010-befa-143c99073115 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:45.316989 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a5f98-4394-46ba-8adb-1e4ddffda165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:45.317650 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a5f98-4394-46ba-8adb-1e4ddffda165 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:45.318051 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a5f98-4394-46ba-8adb-1e4ddffda165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:45.318333 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a5f98-4394-46ba-8adb-1e4ddffda165 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:45.318988 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14a5f98-4394-46ba-8adb-1e4ddffda165 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:45.321024 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14a5f98-4394-46ba-8adb-1e4ddffda165 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:45.427536 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1050/2101] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:07:45 2022] POST /identity/v3/users => generated 298 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:45.431461 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94a07cd-8890-4769-b11d-5d1d91756746 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:45.442243 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94a07cd-8890-4769-b11d-5d1d91756746 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:45.447806 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da017bda-8621-4eb0-a829-7b683ad90b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:45.448450 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da017bda-8621-4eb0-a829-7b683ad90b00 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:45.448782 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da017bda-8621-4eb0-a829-7b683ad90b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:45.449059 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da017bda-8621-4eb0-a829-7b683ad90b00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:45.449755 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da017bda-8621-4eb0-a829-7b683ad90b00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:45.451520 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da017bda-8621-4eb0-a829-7b683ad90b00 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:45.464533 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1052/2102] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:45.469177 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a5f98-4394-46ba-8adb-1e4ddffda165 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:45.486778 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a5f98-4394-46ba-8adb-1e4ddffda165 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:45.492111 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6223b8-7fb4-4e1e-8739-79c99ca9411f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:45.492969 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6223b8-7fb4-4e1e-8739-79c99ca9411f admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:45.493268 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6223b8-7fb4-4e1e-8739-79c99ca9411f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:45.493612 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6223b8-7fb4-4e1e-8739-79c99ca9411f admin admin] PATH_INFO: `/v3/projects/199af3ac334e4da58123ec91fb651a81/users/a7a6200de0124bda87b5d24d82a06b67/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:45.510247 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6223b8-7fb4-4e1e-8739-79c99ca9411f admin admin] RBAC: Authorizing `identity:create_grant(project_id=199af3ac334e4da58123ec91fb651a81, user_id=a7a6200de0124bda87b5d24d82a06b67, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:45.515147 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6223b8-7fb4-4e1e-8739-79c99ca9411f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:45.565176 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:45.591217 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1051/2103] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:45 2022] PUT /identity/v3/projects/199af3ac334e4da58123ec91fb651a81/users/a7a6200de0124bda87b5d24d82a06b67/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:45.595010 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da017bda-8621-4eb0-a829-7b683ad90b00 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:45.616573 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da017bda-8621-4eb0-a829-7b683ad90b00 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:45.678133 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd4773-7c9c-4575-83b0-256ad79bf1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:45.678133 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd4773-7c9c-4575-83b0-256ad79bf1db admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:45.678133 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd4773-7c9c-4575-83b0-256ad79bf1db admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:45.678133 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd4773-7c9c-4575-83b0-256ad79bf1db admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:45.678488 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd4773-7c9c-4575-83b0-256ad79bf1db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:45.678975 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd4773-7c9c-4575-83b0-256ad79bf1db admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:45.688440 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:45.690160 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1053/2104] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:45.696099 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6223b8-7fb4-4e1e-8739-79c99ca9411f admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:45.707501 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6223b8-7fb4-4e1e-8739-79c99ca9411f admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:45.712373 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a21cd4-0981-4d16-b724-9358a9a3da3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:45.713067 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a21cd4-0981-4d16-b724-9358a9a3da3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:45.713350 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a21cd4-0981-4d16-b724-9358a9a3da3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:45.713626 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a21cd4-0981-4d16-b724-9358a9a3da3b admin admin] PATH_INFO: `/v3/projects/199af3ac334e4da58123ec91fb651a81/users/a7a6200de0124bda87b5d24d82a06b67/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:45.715603 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a21cd4-0981-4d16-b724-9358a9a3da3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=199af3ac334e4da58123ec91fb651a81, user_id=a7a6200de0124bda87b5d24d82a06b67, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:45.718343 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a21cd4-0981-4d16-b724-9358a9a3da3b admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:45.729239 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1052/2105] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:45 2022] PUT /identity/v3/projects/199af3ac334e4da58123ec91fb651a81/users/a7a6200de0124bda87b5d24d82a06b67/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:45.741476 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd893a-42e1-4357-a0c0-5b302fa7d250 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:45.741733 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd893a-42e1-4357-a0c0-5b302fa7d250 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:45.742159 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd893a-42e1-4357-a0c0-5b302fa7d250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:45.778251 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5ecd893a-42e1-4357-a0c0-5b302fa7d250 None None] MFA Rules not processed for user `a7a6200de0124bda87b5d24d82a06b67`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:45.779471 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5ecd893a-42e1-4357-a0c0-5b302fa7d250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:45.856492 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1054/2106] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:45 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:45.865904 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a21cd4-0981-4d16-b724-9358a9a3da3b admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:45.867636 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-32a21cd4-0981-4d16-b724-9358a9a3da3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:45.930204 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a21cd4-0981-4d16-b724-9358a9a3da3b admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:45.934486 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfbb6ed-b995-495e-ae29-7a270bec3aae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:45.936879 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfbb6ed-b995-495e-ae29-7a270bec3aae service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:45.937430 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfbb6ed-b995-495e-ae29-7a270bec3aae service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:45.938135 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfbb6ed-b995-495e-ae29-7a270bec3aae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:45.960311 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfbb6ed-b995-495e-ae29-7a270bec3aae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:45.964666 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfbb6ed-b995-495e-ae29-7a270bec3aae service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:45.977339 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1053/2107] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:45 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:07:46.145073 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecd893a-42e1-4357-a0c0-5b302fa7d250 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:46.156498 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecd893a-42e1-4357-a0c0-5b302fa7d250 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:46.161308 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91bda7-dfb2-4d75-aa2b-b736e2813fe7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:46.162069 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91bda7-dfb2-4d75-aa2b-b736e2813fe7 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:46.162469 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91bda7-dfb2-4d75-aa2b-b736e2813fe7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:46.162805 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91bda7-dfb2-4d75-aa2b-b736e2813fe7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:46.264245 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef91bda7-dfb2-4d75-aa2b-b736e2813fe7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:46.265850 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef91bda7-dfb2-4d75-aa2b-b736e2813fe7 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:46.276180 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1055/2108] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:07:46.775819 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfbb6ed-b995-495e-ae29-7a270bec3aae service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:46.807870 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:46.837153 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfbb6ed-b995-495e-ae29-7a270bec3aae service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:46.841697 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961f8d1-6642-4152-a22a-f878fe01f58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:46.842310 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961f8d1-6642-4152-a22a-f878fe01f58f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:46.845489 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961f8d1-6642-4152-a22a-f878fe01f58f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:46.845489 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961f8d1-6642-4152-a22a-f878fe01f58f admin admin] PATH_INFO: `/v3/users/2cdd53c0f23d4fb4bf7e3d68e556dff2` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:46.845489 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6961f8d1-6642-4152-a22a-f878fe01f58f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cdd53c0f23d4fb4bf7e3d68e556dff2)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:46.846473 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6961f8d1-6642-4152-a22a-f878fe01f58f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:46.977970 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:47.073497 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-6961f8d1-6642-4152-a22a-f878fe01f58f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cdd53c0f23d4fb4bf7e3d68e556dff2'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:47.074285 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-6961f8d1-6642-4152-a22a-f878fe01f58f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cdd53c0f23d4fb4bf7e3d68e556dff2'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:47.152898 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-6961f8d1-6642-4152-a22a-f878fe01f58f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cdd53c0f23d4fb4bf7e3d68e556dff2'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:47.153599 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-6961f8d1-6642-4152-a22a-f878fe01f58f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cdd53c0f23d4fb4bf7e3d68e556dff2'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:47.167426 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-6961f8d1-6642-4152-a22a-f878fe01f58f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cdd53c0f23d4fb4bf7e3d68e556dff2'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:47.182983 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1054/2109] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:07:46 2022] DELETE /identity/v3/users/2cdd53c0f23d4fb4bf7e3d68e556dff2 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:47.187557 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21363c8-3c5b-4331-80db-78e73f45cca9 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:47.188094 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21363c8-3c5b-4331-80db-78e73f45cca9 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:47.188185 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21363c8-3c5b-4331-80db-78e73f45cca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:47.209171 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-b21363c8-3c5b-4331-80db-78e73f45cca9 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:47.211273 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b21363c8-3c5b-4331-80db-78e73f45cca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:47.292360 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1056/2110] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:47.302386 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961f8d1-6642-4152-a22a-f878fe01f58f admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:47.303790 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-6961f8d1-6642-4152-a22a-f878fe01f58f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:47.364938 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961f8d1-6642-4152-a22a-f878fe01f58f admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:47.369668 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8297a28e-e98d-4660-a989-89dc466334e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:47.370468 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297a28e-e98d-4660-a989-89dc466334e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:47.370985 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297a28e-e98d-4660-a989-89dc466334e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:47.371426 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297a28e-e98d-4660-a989-89dc466334e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:47.382679 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:47.383333 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8297a28e-e98d-4660-a989-89dc466334e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:47.384853 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8297a28e-e98d-4660-a989-89dc466334e8 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:47.395945 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1055/2111] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:07:47.494378 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:47.813245 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:47.951737 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21363c8-3c5b-4331-80db-78e73f45cca9 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:47.953191 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b21363c8-3c5b-4331-80db-78e73f45cca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:47.968855 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21363c8-3c5b-4331-80db-78e73f45cca9 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:47.974757 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885eb0ba-e6e4-41a4-9d05-f20732b02362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:47.975397 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885eb0ba-e6e4-41a4-9d05-f20732b02362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:47.975692 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885eb0ba-e6e4-41a4-9d05-f20732b02362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:47.975966 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885eb0ba-e6e4-41a4-9d05-f20732b02362 admin admin] PATH_INFO: `/v3/projects/924b7ec4bd73415aa64de7389fe4c004` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:47.977183 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885eb0ba-e6e4-41a4-9d05-f20732b02362 admin admin] RBAC: Authorizing `identity:delete_project(project_id=924b7ec4bd73415aa64de7389fe4c004)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:47.983627 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885eb0ba-e6e4-41a4-9d05-f20732b02362 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:48.042472 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8297a28e-e98d-4660-a989-89dc466334e8 service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:48.043021 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8297a28e-e98d-4660-a989-89dc466334e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:48.056697 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8297a28e-e98d-4660-a989-89dc466334e8 service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:48.061797 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca82908b-e468-4b2b-8f3c-cc7ed7bb61fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:48.063353 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82908b-e468-4b2b-8f3c-cc7ed7bb61fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:48.063643 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82908b-e468-4b2b-8f3c-cc7ed7bb61fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:48.063921 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82908b-e468-4b2b-8f3c-cc7ed7bb61fc admin admin] PATH_INFO: `/v3/users/74e7b6f4cd47468e9e5ea62befe4abf4` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:48.065274 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca82908b-e468-4b2b-8f3c-cc7ed7bb61fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=74e7b6f4cd47468e9e5ea62befe4abf4)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:48.069318 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca82908b-e468-4b2b-8f3c-cc7ed7bb61fc admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:48.107010 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:48.123899 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-885eb0ba-e6e4-41a4-9d05-f20732b02362 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '924b7ec4bd73415aa64de7389fe4c004', 'request_id': 'req-885eb0ba-e6e4-41a4-9d05-f20732b02362'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:48.204230 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-885eb0ba-e6e4-41a4-9d05-f20732b02362 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '924b7ec4bd73415aa64de7389fe4c004', 'request_id': 'req-885eb0ba-e6e4-41a4-9d05-f20732b02362'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:48.211028 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-885eb0ba-e6e4-41a4-9d05-f20732b02362 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '924b7ec4bd73415aa64de7389fe4c004', 'request_id': 'req-885eb0ba-e6e4-41a4-9d05-f20732b02362'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:48.222921 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-885eb0ba-e6e4-41a4-9d05-f20732b02362 admin admin] The token cache is being invalidate because project 924b7ec4bd73415aa64de7389fe4c004 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:07:48.223186 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-885eb0ba-e6e4-41a4-9d05-f20732b02362 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:48.223847 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-885eb0ba-e6e4-41a4-9d05-f20732b02362 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:48.226002 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1057/2112] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:07:47 2022] DELETE /identity/v3/projects/924b7ec4bd73415aa64de7389fe4c004 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:48.230807 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885eb0ba-e6e4-41a4-9d05-f20732b02362 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:48.232237 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-885eb0ba-e6e4-41a4-9d05-f20732b02362 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:48.241710 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ca82908b-e468-4b2b-8f3c-cc7ed7bb61fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74e7b6f4cd47468e9e5ea62befe4abf4'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:48.242646 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ca82908b-e468-4b2b-8f3c-cc7ed7bb61fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74e7b6f4cd47468e9e5ea62befe4abf4'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:48.304249 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885eb0ba-e6e4-41a4-9d05-f20732b02362 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:48.309033 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bef58e-87f9-4e44-ada6-13097f15ac26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:48.309669 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bef58e-87f9-4e44-ada6-13097f15ac26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:48.309927 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bef58e-87f9-4e44-ada6-13097f15ac26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:48.310209 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bef58e-87f9-4e44-ada6-13097f15ac26 admin admin] PATH_INFO: `/v3/users/a7a6200de0124bda87b5d24d82a06b67` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:48.311580 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bef58e-87f9-4e44-ada6-13097f15ac26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7a6200de0124bda87b5d24d82a06b67)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:48.313279 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ca82908b-e468-4b2b-8f3c-cc7ed7bb61fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74e7b6f4cd47468e9e5ea62befe4abf4'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:48.313502 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bef58e-87f9-4e44-ada6-13097f15ac26 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:48.314065 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ca82908b-e468-4b2b-8f3c-cc7ed7bb61fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74e7b6f4cd47468e9e5ea62befe4abf4'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:48.326863 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ca82908b-e468-4b2b-8f3c-cc7ed7bb61fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74e7b6f4cd47468e9e5ea62befe4abf4'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:48.340602 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1056/2113] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:07:48 2022] DELETE /identity/v3/users/74e7b6f4cd47468e9e5ea62befe4abf4 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:07:48.344961 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac1307-f0aa-4df6-b03b-16226b5200bc None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:48.345270 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac1307-f0aa-4df6-b03b-16226b5200bc None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:48.350281 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac1307-f0aa-4df6-b03b-16226b5200bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:48.366635 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-f3ac1307-f0aa-4df6-b03b-16226b5200bc None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:48.367947 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f3ac1307-f0aa-4df6-b03b-16226b5200bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:48.436827 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1057/2114] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:48.447777 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac1307-f0aa-4df6-b03b-16226b5200bc None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:48.449345 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f3ac1307-f0aa-4df6-b03b-16226b5200bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:48.513677 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac1307-f0aa-4df6-b03b-16226b5200bc None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:48.518642 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e10610-16b0-4253-87eb-e401cd988bbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:48.519954 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e10610-16b0-4253-87eb-e401cd988bbd service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:48.520451 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e10610-16b0-4253-87eb-e401cd988bbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:48.520796 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e10610-16b0-4253-87eb-e401cd988bbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:48.532086 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e10610-16b0-4253-87eb-e401cd988bbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:48.533567 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e10610-16b0-4253-87eb-e401cd988bbd service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:48.535192 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d2bef58e-87f9-4e44-ada6-13097f15ac26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7a6200de0124bda87b5d24d82a06b67'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:48.536128 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d2bef58e-87f9-4e44-ada6-13097f15ac26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7a6200de0124bda87b5d24d82a06b67'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:48.543988 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d2bef58e-87f9-4e44-ada6-13097f15ac26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7a6200de0124bda87b5d24d82a06b67'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:48.548234 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1058/2115] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 01:07:48.621812 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d2bef58e-87f9-4e44-ada6-13097f15ac26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7a6200de0124bda87b5d24d82a06b67'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:48.622515 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d2bef58e-87f9-4e44-ada6-13097f15ac26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7a6200de0124bda87b5d24d82a06b67'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:48.656088 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1058/2116] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:07:48 2022] DELETE /identity/v3/users/a7a6200de0124bda87b5d24d82a06b67 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:48.767938 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:48.814079 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:49.021384 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:49.110525 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e10610-16b0-4253-87eb-e401cd988bbd service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:49.113019 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c4e10610-16b0-4253-87eb-e401cd988bbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:49.151133 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e10610-16b0-4253-87eb-e401cd988bbd service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:49.159139 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:49.247282 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:49.252386 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb1792-b70c-48d3-aa55-c27c8367618f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:49.253070 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb1792-b70c-48d3-aa55-c27c8367618f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:49.254110 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb1792-b70c-48d3-aa55-c27c8367618f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:49.254541 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb1792-b70c-48d3-aa55-c27c8367618f admin admin] PATH_INFO: `/v3/projects/1c9bcb20d46a4b26a9df04a0467dc063` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:49.255803 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb1792-b70c-48d3-aa55-c27c8367618f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c9bcb20d46a4b26a9df04a0467dc063)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:49.258366 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb1792-b70c-48d3-aa55-c27c8367618f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:49.409846 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bef58e-87f9-4e44-ada6-13097f15ac26 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:49.411397 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d2bef58e-87f9-4e44-ada6-13097f15ac26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:49.428531 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bef58e-87f9-4e44-ada6-13097f15ac26 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:49.452558 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-30fb1792-b70c-48d3-aa55-c27c8367618f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c9bcb20d46a4b26a9df04a0467dc063', 'request_id': 'req-30fb1792-b70c-48d3-aa55-c27c8367618f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:49.485834 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:49.496584 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b4b735-c82d-48c6-8e8c-b31420e77b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:49.497234 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b4b735-c82d-48c6-8e8c-b31420e77b48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:49.497519 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b4b735-c82d-48c6-8e8c-b31420e77b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:49.497798 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b4b735-c82d-48c6-8e8c-b31420e77b48 admin admin] PATH_INFO: `/v3/projects/199af3ac334e4da58123ec91fb651a81` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:49.498967 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b4b735-c82d-48c6-8e8c-b31420e77b48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=199af3ac334e4da58123ec91fb651a81)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:49.501006 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b4b735-c82d-48c6-8e8c-b31420e77b48 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:49.513643 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-30fb1792-b70c-48d3-aa55-c27c8367618f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c9bcb20d46a4b26a9df04a0467dc063', 'request_id': 'req-30fb1792-b70c-48d3-aa55-c27c8367618f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:49.522123 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-30fb1792-b70c-48d3-aa55-c27c8367618f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c9bcb20d46a4b26a9df04a0467dc063', 'request_id': 'req-30fb1792-b70c-48d3-aa55-c27c8367618f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:49.534211 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-30fb1792-b70c-48d3-aa55-c27c8367618f admin admin] The token cache is being invalidate because project 1c9bcb20d46a4b26a9df04a0467dc063 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:07:49.534547 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-30fb1792-b70c-48d3-aa55-c27c8367618f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:49.535215 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-30fb1792-b70c-48d3-aa55-c27c8367618f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:49.536743 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1059/2117] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:07:49 2022] DELETE /identity/v3/projects/1c9bcb20d46a4b26a9df04a0467dc063 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:49.540739 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb1792-b70c-48d3-aa55-c27c8367618f admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:49.542359 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-30fb1792-b70c-48d3-aa55-c27c8367618f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:49.555512 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb1792-b70c-48d3-aa55-c27c8367618f admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:49.561098 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7990721e-d23c-4a95-9cbc-e0a08da510d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:49.561775 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990721e-d23c-4a95-9cbc-e0a08da510d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:49.562053 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990721e-d23c-4a95-9cbc-e0a08da510d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:49.562325 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990721e-d23c-4a95-9cbc-e0a08da510d9 admin admin] PATH_INFO: `/v3/users/7d8b4b0cf5c24129b6683051682aed55` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:49.563494 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7990721e-d23c-4a95-9cbc-e0a08da510d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d8b4b0cf5c24129b6683051682aed55)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:49.565455 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7990721e-d23c-4a95-9cbc-e0a08da510d9 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:49.633652 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b6b4b735-c82d-48c6-8e8c-b31420e77b48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '199af3ac334e4da58123ec91fb651a81', 'request_id': 'req-b6b4b735-c82d-48c6-8e8c-b31420e77b48'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:49.644777 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:49.712975 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b6b4b735-c82d-48c6-8e8c-b31420e77b48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '199af3ac334e4da58123ec91fb651a81', 'request_id': 'req-b6b4b735-c82d-48c6-8e8c-b31420e77b48'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:49.719392 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b6b4b735-c82d-48c6-8e8c-b31420e77b48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '199af3ac334e4da58123ec91fb651a81', 'request_id': 'req-b6b4b735-c82d-48c6-8e8c-b31420e77b48'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:49.731876 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b6b4b735-c82d-48c6-8e8c-b31420e77b48 admin admin] The token cache is being invalidate because project 199af3ac334e4da58123ec91fb651a81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:07:49.732172 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b6b4b735-c82d-48c6-8e8c-b31420e77b48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:49.732893 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b6b4b735-c82d-48c6-8e8c-b31420e77b48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:49.734481 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1059/2118] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:07:49 2022] DELETE /identity/v3/projects/199af3ac334e4da58123ec91fb651a81 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:07:49.739171 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60085275-baf7-4a96-b6a1-5cbf07a6b91c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:49.739607 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60085275-baf7-4a96-b6a1-5cbf07a6b91c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:49.739810 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60085275-baf7-4a96-b6a1-5cbf07a6b91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:49.759260 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-60085275-baf7-4a96-b6a1-5cbf07a6b91c None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:49.760604 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-60085275-baf7-4a96-b6a1-5cbf07a6b91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:49.773056 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-7990721e-d23c-4a95-9cbc-e0a08da510d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d8b4b0cf5c24129b6683051682aed55'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:49.774060 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-7990721e-d23c-4a95-9cbc-e0a08da510d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d8b4b0cf5c24129b6683051682aed55'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:49.777640 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:49.829098 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1060/2119] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:07:49.831182 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb227c-fadd-4f36-8e82-d2df67716c28 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:49.832282 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb227c-fadd-4f36-8e82-d2df67716c28 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:49.832560 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb227c-fadd-4f36-8e82-d2df67716c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:49.857149 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-bebb227c-fadd-4f36-8e82-d2df67716c28 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:49.859348 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bebb227c-fadd-4f36-8e82-d2df67716c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:49.861282 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-7990721e-d23c-4a95-9cbc-e0a08da510d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d8b4b0cf5c24129b6683051682aed55'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:49.861969 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-7990721e-d23c-4a95-9cbc-e0a08da510d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d8b4b0cf5c24129b6683051682aed55'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:49.868804 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1061/2120] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:49.873344 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb227c-fadd-4f36-8e82-d2df67716c28 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:49.876158 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-7990721e-d23c-4a95-9cbc-e0a08da510d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d8b4b0cf5c24129b6683051682aed55'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:49.883603 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb227c-fadd-4f36-8e82-d2df67716c28 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:49.889006 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4da04e-5aad-476c-8dc5-3d4cbc1bc1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:49.889607 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4da04e-5aad-476c-8dc5-3d4cbc1bc1f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:49.889888 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4da04e-5aad-476c-8dc5-3d4cbc1bc1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:49.890165 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4da04e-5aad-476c-8dc5-3d4cbc1bc1f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:49.890814 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4da04e-5aad-476c-8dc5-3d4cbc1bc1f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:07:49.891121 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4da04e-5aad-476c-8dc5-3d4cbc1bc1f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:49.892740 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4da04e-5aad-476c-8dc5-3d4cbc1bc1f2 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:49.894096 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1060/2121] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:07:49 2022] DELETE /identity/v3/users/7d8b4b0cf5c24129b6683051682aed55 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:07:49.905275 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1062/2122] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:07:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:07:49.909751 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7990721e-d23c-4a95-9cbc-e0a08da510d9 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:49.999670 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7990721e-d23c-4a95-9cbc-e0a08da510d9 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:50.005329 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50560e17-1ad7-44a2-8097-9de1f99fc116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:50.006253 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50560e17-1ad7-44a2-8097-9de1f99fc116 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:50.006543 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50560e17-1ad7-44a2-8097-9de1f99fc116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:50.011749 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50560e17-1ad7-44a2-8097-9de1f99fc116 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:50.011749 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50560e17-1ad7-44a2-8097-9de1f99fc116 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:50.011749 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50560e17-1ad7-44a2-8097-9de1f99fc116 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:50.096796 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1061/2123] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:07:49 2022] POST /identity/v3/projects => generated 373 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:50.101205 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4da04e-5aad-476c-8dc5-3d4cbc1bc1f2 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:50.120929 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4da04e-5aad-476c-8dc5-3d4cbc1bc1f2 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:50.173472 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:50.189207 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77fb019-1ff7-4128-aa8c-6f2cb156c198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:50.189905 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fb019-1ff7-4128-aa8c-6f2cb156c198 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:50.190195 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fb019-1ff7-4128-aa8c-6f2cb156c198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:50.190470 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fb019-1ff7-4128-aa8c-6f2cb156c198 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:50.191170 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77fb019-1ff7-4128-aa8c-6f2cb156c198 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:50.193321 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77fb019-1ff7-4128-aa8c-6f2cb156c198 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:50.311122 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1063/2124] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:07:50 2022] POST /identity/v3/users => generated 287 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:50.315303 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50560e17-1ad7-44a2-8097-9de1f99fc116 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:50.326153 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50560e17-1ad7-44a2-8097-9de1f99fc116 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:50.331941 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a34dae4-6032-4a29-baaf-f8822ba39175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:50.332963 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34dae4-6032-4a29-baaf-f8822ba39175 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:50.333364 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34dae4-6032-4a29-baaf-f8822ba39175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:50.333721 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34dae4-6032-4a29-baaf-f8822ba39175 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:50.335070 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a34dae4-6032-4a29-baaf-f8822ba39175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:50.336845 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a34dae4-6032-4a29-baaf-f8822ba39175 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:50.350575 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1062/2125] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:50.354218 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77fb019-1ff7-4128-aa8c-6f2cb156c198 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:50.365659 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77fb019-1ff7-4128-aa8c-6f2cb156c198 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:50.375853 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5dfb11-b18e-4dd9-aab1-a8911be22cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:50.377196 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5dfb11-b18e-4dd9-aab1-a8911be22cad admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:50.377196 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5dfb11-b18e-4dd9-aab1-a8911be22cad admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:50.377416 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5dfb11-b18e-4dd9-aab1-a8911be22cad admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:50.378100 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5dfb11-b18e-4dd9-aab1-a8911be22cad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:50.380310 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5dfb11-b18e-4dd9-aab1-a8911be22cad admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:50.397321 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1064/2126] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:50 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 01:07:50.403735 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a34dae4-6032-4a29-baaf-f8822ba39175 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:50.414567 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5dfb11-b18e-4dd9-aab1-a8911be22cad admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:50.419005 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7e5dfb11-b18e-4dd9-aab1-a8911be22cad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:50.438756 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5dfb11-b18e-4dd9-aab1-a8911be22cad admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:50.443905 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a34dae4-6032-4a29-baaf-f8822ba39175 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:50.447235 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d57fbb-68b5-472f-846d-6b771d04b52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:50.447235 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d57fbb-68b5-472f-846d-6b771d04b52a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:50.454589 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d57fbb-68b5-472f-846d-6b771d04b52a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:50.454589 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d57fbb-68b5-472f-846d-6b771d04b52a admin admin] PATH_INFO: `/v3/projects/587d827123c44842b575b43d967d8b68` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:50.456136 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d57fbb-68b5-472f-846d-6b771d04b52a admin admin] RBAC: Authorizing `identity:delete_project(project_id=587d827123c44842b575b43d967d8b68)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:50.458867 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d57fbb-68b5-472f-846d-6b771d04b52a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:50.471779 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b264c62-b001-4eaa-93a8-0fdb47788bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:50.473755 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b264c62-b001-4eaa-93a8-0fdb47788bfc admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:50.474046 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b264c62-b001-4eaa-93a8-0fdb47788bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:50.474324 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b264c62-b001-4eaa-93a8-0fdb47788bfc admin admin] PATH_INFO: `/v3/projects/3ece345f5fb34b3cbdb32d2880bf67da/users/7a1fc708e8cd4b42baa8a661a625606b/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:50.499443 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b264c62-b001-4eaa-93a8-0fdb47788bfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ece345f5fb34b3cbdb32d2880bf67da, user_id=7a1fc708e8cd4b42baa8a661a625606b, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:50.505252 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b264c62-b001-4eaa-93a8-0fdb47788bfc admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:50.553449 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:50.618293 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1063/2127] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:50 2022] PUT /identity/v3/projects/3ece345f5fb34b3cbdb32d2880bf67da/users/7a1fc708e8cd4b42baa8a661a625606b/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:50.624871 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24555368-5aa7-4d8a-a650-bf3714eb01f4 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:50.625385 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24555368-5aa7-4d8a-a650-bf3714eb01f4 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:50.625934 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24555368-5aa7-4d8a-a650-bf3714eb01f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:50.630609 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a3d57fbb-68b5-472f-846d-6b771d04b52a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '587d827123c44842b575b43d967d8b68', 'request_id': 'req-a3d57fbb-68b5-472f-846d-6b771d04b52a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:50.655590 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-24555368-5aa7-4d8a-a650-bf3714eb01f4 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:50.656888 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-24555368-5aa7-4d8a-a650-bf3714eb01f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:50.719456 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1064/2128] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:50.730097 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24555368-5aa7-4d8a-a650-bf3714eb01f4 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:50.730258 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a3d57fbb-68b5-472f-846d-6b771d04b52a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '587d827123c44842b575b43d967d8b68', 'request_id': 'req-a3d57fbb-68b5-472f-846d-6b771d04b52a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:50.731828 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-24555368-5aa7-4d8a-a650-bf3714eb01f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:50.736634 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a3d57fbb-68b5-472f-846d-6b771d04b52a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '587d827123c44842b575b43d967d8b68', 'request_id': 'req-a3d57fbb-68b5-472f-846d-6b771d04b52a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:50.749987 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a3d57fbb-68b5-472f-846d-6b771d04b52a admin admin] The token cache is being invalidate because project 587d827123c44842b575b43d967d8b68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:07:50.750636 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a3d57fbb-68b5-472f-846d-6b771d04b52a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:50.751617 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a3d57fbb-68b5-472f-846d-6b771d04b52a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:50.754126 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1065/2129] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:07:50 2022] DELETE /identity/v3/projects/587d827123c44842b575b43d967d8b68 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:50.760113 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29a508-5848-46d3-9a59-2b5f48282fad None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:50.763082 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29a508-5848-46d3-9a59-2b5f48282fad None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:50.763082 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29a508-5848-46d3-9a59-2b5f48282fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:50.791218 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-9c29a508-5848-46d3-9a59-2b5f48282fad None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:50.792547 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9c29a508-5848-46d3-9a59-2b5f48282fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:50.802292 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1066/2130] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:07:50.808178 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24555368-5aa7-4d8a-a650-bf3714eb01f4 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:50.810982 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8c62d-a884-4979-8a30-f5a1cc98cad2 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:50.811274 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8c62d-a884-4979-8a30-f5a1cc98cad2 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:50.811558 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8c62d-a884-4979-8a30-f5a1cc98cad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:50.812270 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd5a729-2ad9-40ec-aef2-d88c7c5e2dd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:50.813110 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd5a729-2ad9-40ec-aef2-d88c7c5e2dd5 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:50.813539 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd5a729-2ad9-40ec-aef2-d88c7c5e2dd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:50.813960 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd5a729-2ad9-40ec-aef2-d88c7c5e2dd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:50.816402 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-4fd5a729-2ad9-40ec-aef2-d88c7c5e2dd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:50.830047 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd5a729-2ad9-40ec-aef2-d88c7c5e2dd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:50.832379 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd5a729-2ad9-40ec-aef2-d88c7c5e2dd5 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:50.834365 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c8f8c62d-a884-4979-8a30-f5a1cc98cad2 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:50.835588 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c8f8c62d-a884-4979-8a30-f5a1cc98cad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:50.843562 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1065/2131] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:07:50.844311 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1067/2132] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:50.848523 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd5a729-2ad9-40ec-aef2-d88c7c5e2dd5 service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:50.859415 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd5a729-2ad9-40ec-aef2-d88c7c5e2dd5 service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:50.864213 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43834de3-bb33-4089-a35f-ec7628b30759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:50.864856 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43834de3-bb33-4089-a35f-ec7628b30759 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:50.865183 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43834de3-bb33-4089-a35f-ec7628b30759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:50.865471 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43834de3-bb33-4089-a35f-ec7628b30759 admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:50.866181 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43834de3-bb33-4089-a35f-ec7628b30759 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:07:50.866422 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43834de3-bb33-4089-a35f-ec7628b30759 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:50.868064 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43834de3-bb33-4089-a35f-ec7628b30759 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:50.879382 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:50.883183 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1066/2133] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:07:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 01:07:50.889149 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f8c62d-a884-4979-8a30-f5a1cc98cad2 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:50.899561 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f8c62d-a884-4979-8a30-f5a1cc98cad2 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:50.905654 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4689cb25-d7cb-402b-b118-2454dd2636d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:50.906378 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689cb25-d7cb-402b-b118-2454dd2636d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:50.906718 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689cb25-d7cb-402b-b118-2454dd2636d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:50.907595 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689cb25-d7cb-402b-b118-2454dd2636d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:50.908431 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4689cb25-d7cb-402b-b118-2454dd2636d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:50.912229 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4689cb25-d7cb-402b-b118-2454dd2636d2 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:51.024379 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1068/2134] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:07:50 2022] POST /identity/v3/projects => generated 377 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:51.028268 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43834de3-bb33-4089-a35f-ec7628b30759 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:51.040071 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43834de3-bb33-4089-a35f-ec7628b30759 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:51.108277 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b4aee-29b4-4d64-ae5e-e71ff1f46199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:51.109049 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b4aee-29b4-4d64-ae5e-e71ff1f46199 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:51.109290 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b4aee-29b4-4d64-ae5e-e71ff1f46199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:51.109717 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b4aee-29b4-4d64-ae5e-e71ff1f46199 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:51.110361 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3b4aee-29b4-4d64-ae5e-e71ff1f46199 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:51.112410 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3b4aee-29b4-4d64-ae5e-e71ff1f46199 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:51.204023 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1067/2135] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:07:51 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:07:51.208731 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4689cb25-d7cb-402b-b118-2454dd2636d2 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:51.220991 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4689cb25-d7cb-402b-b118-2454dd2636d2 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:51.231422 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a26f2e3-0882-47ca-8973-146754761a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:51.232159 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26f2e3-0882-47ca-8973-146754761a3a admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:51.232383 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26f2e3-0882-47ca-8973-146754761a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:51.234644 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26f2e3-0882-47ca-8973-146754761a3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:51.234644 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a26f2e3-0882-47ca-8973-146754761a3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:51.235087 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a26f2e3-0882-47ca-8973-146754761a3a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:51.246712 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1069/2136] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:51.250572 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b4aee-29b4-4d64-ae5e-e71ff1f46199 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:51.263700 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b4aee-29b4-4d64-ae5e-e71ff1f46199 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:51.269619 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebfe082-f90b-425d-9772-28731f99e784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:51.270501 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebfe082-f90b-425d-9772-28731f99e784 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:51.270929 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebfe082-f90b-425d-9772-28731f99e784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:51.271345 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebfe082-f90b-425d-9772-28731f99e784 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:51.272665 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebfe082-f90b-425d-9772-28731f99e784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:51.274755 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebfe082-f90b-425d-9772-28731f99e784 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:51.288393 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1068/2137] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 01:07:51.293147 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a26f2e3-0882-47ca-8973-146754761a3a admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:51.305209 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a26f2e3-0882-47ca-8973-146754761a3a admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:51.310436 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e9960-760f-4c6a-beb4-a738d754679c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:51.311219 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e9960-760f-4c6a-beb4-a738d754679c admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:51.311599 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e9960-760f-4c6a-beb4-a738d754679c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:51.311931 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e9960-760f-4c6a-beb4-a738d754679c admin admin] PATH_INFO: `/v3/projects/ceded6b844164be194c5d6efbb1d3b90/users/fc3983ab83e24d3784e79fd501956191/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:51.328632 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581e9960-760f-4c6a-beb4-a738d754679c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceded6b844164be194c5d6efbb1d3b90, user_id=fc3983ab83e24d3784e79fd501956191, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:51.331670 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581e9960-760f-4c6a-beb4-a738d754679c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:51.464054 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1070/2138] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:51 2022] PUT /identity/v3/projects/ceded6b844164be194c5d6efbb1d3b90/users/fc3983ab83e24d3784e79fd501956191/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:07:51.475948 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e38cc-9eed-4518-8a4f-1ed2a0514581 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:51.476395 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e38cc-9eed-4518-8a4f-1ed2a0514581 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:51.477059 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e38cc-9eed-4518-8a4f-1ed2a0514581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:51.511331 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-764e38cc-9eed-4518-8a4f-1ed2a0514581 None None] MFA Rules not processed for user `fc3983ab83e24d3784e79fd501956191`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:51.512937 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-764e38cc-9eed-4518-8a4f-1ed2a0514581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:51.579572 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1069/2139] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:51.584553 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e9960-760f-4c6a-beb4-a738d754679c admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:51.596548 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:51.649299 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e9960-760f-4c6a-beb4-a738d754679c admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:51.654406 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f3280-e84e-4955-a2ce-6ac5c7d371d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:51.654947 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f3280-e84e-4955-a2ce-6ac5c7d371d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:51.655217 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f3280-e84e-4955-a2ce-6ac5c7d371d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:51.655490 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f3280-e84e-4955-a2ce-6ac5c7d371d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:51.656124 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f3280-e84e-4955-a2ce-6ac5c7d371d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:51.658259 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f3280-e84e-4955-a2ce-6ac5c7d371d0 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:51.779487 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1071/2140] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:07:51 2022] POST /identity/v3/projects => generated 375 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:51.783570 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764e38cc-9eed-4518-8a4f-1ed2a0514581 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:51.800132 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764e38cc-9eed-4518-8a4f-1ed2a0514581 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:51.873306 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5744c6f-f6bd-48d1-a79b-af4c632f6cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:51.873926 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5744c6f-f6bd-48d1-a79b-af4c632f6cf3 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:51.874208 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5744c6f-f6bd-48d1-a79b-af4c632f6cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:51.874483 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5744c6f-f6bd-48d1-a79b-af4c632f6cf3 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:51.875839 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5744c6f-f6bd-48d1-a79b-af4c632f6cf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:51.877924 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5744c6f-f6bd-48d1-a79b-af4c632f6cf3 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:51.971238 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1070/2141] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:07:51 2022] POST /identity/v3/users => generated 288 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:51.976084 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f3280-e84e-4955-a2ce-6ac5c7d371d0 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:51.987182 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f3280-e84e-4955-a2ce-6ac5c7d371d0 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:51.993603 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df51c9a4-7045-4241-86d1-48d46b8dfa86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:51.994220 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51c9a4-7045-4241-86d1-48d46b8dfa86 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:51.994503 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51c9a4-7045-4241-86d1-48d46b8dfa86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:51.994780 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51c9a4-7045-4241-86d1-48d46b8dfa86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:51.995423 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df51c9a4-7045-4241-86d1-48d46b8dfa86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:51.997886 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df51c9a4-7045-4241-86d1-48d46b8dfa86 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:52.011495 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1072/2142] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 01:07:52.013353 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5744c6f-f6bd-48d1-a79b-af4c632f6cf3 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:52.025722 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5744c6f-f6bd-48d1-a79b-af4c632f6cf3 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:52.031750 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94265f1-c6aa-4f60-97d7-dd527869ec51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:52.032494 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94265f1-c6aa-4f60-97d7-dd527869ec51 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:52.032838 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94265f1-c6aa-4f60-97d7-dd527869ec51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:52.033193 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94265f1-c6aa-4f60-97d7-dd527869ec51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:52.033897 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94265f1-c6aa-4f60-97d7-dd527869ec51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:52.036665 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94265f1-c6aa-4f60-97d7-dd527869ec51 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:52.047916 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1071/2143] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 01:07:52.053163 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df51c9a4-7045-4241-86d1-48d46b8dfa86 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:52.064306 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df51c9a4-7045-4241-86d1-48d46b8dfa86 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:52.070122 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273643d-ff9c-40d7-b894-6c399b07f496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:52.070790 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273643d-ff9c-40d7-b894-6c399b07f496 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:52.071069 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273643d-ff9c-40d7-b894-6c399b07f496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:52.071342 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273643d-ff9c-40d7-b894-6c399b07f496 admin admin] PATH_INFO: `/v3/projects/53a25412091c4285a3124433e04f4464/users/ab7fddc495c04c3084add4ede84fbaea/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:52.093440 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6273643d-ff9c-40d7-b894-6c399b07f496 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53a25412091c4285a3124433e04f4464, user_id=ab7fddc495c04c3084add4ede84fbaea, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:52.093440 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6273643d-ff9c-40d7-b894-6c399b07f496 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:52.177466 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1073/2144] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:52 2022] PUT /identity/v3/projects/53a25412091c4285a3124433e04f4464/users/ab7fddc495c04c3084add4ede84fbaea/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:52.191387 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96360ad7-50b7-4aff-ada3-39adee8d4299 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:52.192788 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96360ad7-50b7-4aff-ada3-39adee8d4299 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:52.193092 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96360ad7-50b7-4aff-ada3-39adee8d4299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:52.228119 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-96360ad7-50b7-4aff-ada3-39adee8d4299 None None] MFA Rules not processed for user `ab7fddc495c04c3084add4ede84fbaea`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:52.230185 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-96360ad7-50b7-4aff-ada3-39adee8d4299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:52.266038 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:52.335328 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1072/2145] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:52.339811 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273643d-ff9c-40d7-b894-6c399b07f496 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:52.418562 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273643d-ff9c-40d7-b894-6c399b07f496 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:52.423411 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8eb98-4880-4d30-966b-a6b6613d003a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:52.424017 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8eb98-4880-4d30-966b-a6b6613d003a admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:52.428957 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8eb98-4880-4d30-966b-a6b6613d003a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:52.428957 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8eb98-4880-4d30-966b-a6b6613d003a admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:52.428957 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8eb98-4880-4d30-966b-a6b6613d003a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:52.428957 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8eb98-4880-4d30-966b-a6b6613d003a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:52.555142 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1074/2146] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:07:52 2022] POST /identity/v3/projects => generated 377 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:52.559540 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96360ad7-50b7-4aff-ada3-39adee8d4299 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:52.579847 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96360ad7-50b7-4aff-ada3-39adee8d4299 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:52.592936 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9aab0-0995-49ac-9a92-d91c8115ad62 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:52.593303 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9aab0-0995-49ac-9a92-d91c8115ad62 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:52.593658 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9aab0-0995-49ac-9a92-d91c8115ad62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:52.668997 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-33c9aab0-0995-49ac-9a92-d91c8115ad62 None None] MFA Rules not processed for user `7a1fc708e8cd4b42baa8a661a625606b`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:52.672204 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-33c9aab0-0995-49ac-9a92-d91c8115ad62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:52.752799 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce79f5f8-cec3-48fe-9fde-3ec39ac58246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:52.754237 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79f5f8-cec3-48fe-9fde-3ec39ac58246 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:52.754612 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79f5f8-cec3-48fe-9fde-3ec39ac58246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:52.754891 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79f5f8-cec3-48fe-9fde-3ec39ac58246 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:52.755556 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce79f5f8-cec3-48fe-9fde-3ec39ac58246 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:52.757576 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce79f5f8-cec3-48fe-9fde-3ec39ac58246 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:52.773227 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1075/2147] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:52.774425 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9aab0-0995-49ac-9a92-d91c8115ad62 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:52.776057 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-33c9aab0-0995-49ac-9a92-d91c8115ad62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:52.792398 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9aab0-0995-49ac-9a92-d91c8115ad62 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:52.797586 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6209789-7946-4870-bea2-2ba820d6c9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:52.798242 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6209789-7946-4870-bea2-2ba820d6c9ee admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:52.798536 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6209789-7946-4870-bea2-2ba820d6c9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:52.798820 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6209789-7946-4870-bea2-2ba820d6c9ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:52.799554 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6209789-7946-4870-bea2-2ba820d6c9ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:52.801634 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6209789-7946-4870-bea2-2ba820d6c9ee admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:52.804744 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:52.888418 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1073/2148] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:07:52 2022] POST /identity/v3/users => generated 295 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:52.892554 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce79f5f8-cec3-48fe-9fde-3ec39ac58246 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:52.902630 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce79f5f8-cec3-48fe-9fde-3ec39ac58246 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:52.907476 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e34250-474d-47df-af70-67e9279ec606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:52.908074 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e34250-474d-47df-af70-67e9279ec606 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:52.908361 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e34250-474d-47df-af70-67e9279ec606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:52.908817 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e34250-474d-47df-af70-67e9279ec606 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:52.910017 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e34250-474d-47df-af70-67e9279ec606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:52.912628 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e34250-474d-47df-af70-67e9279ec606 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:52.923187 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1074/2149] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:52.928038 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e34250-474d-47df-af70-67e9279ec606 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:52.932609 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1076/2150] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:07:52 2022] POST /identity/v3/projects => generated 371 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:52.936692 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6209789-7946-4870-bea2-2ba820d6c9ee admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:52.942501 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e34250-474d-47df-af70-67e9279ec606 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:53.008104 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6209789-7946-4870-bea2-2ba820d6c9ee admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:53.013338 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a229ab-b507-4219-98cc-3c3150cba595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:53.013948 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a229ab-b507-4219-98cc-3c3150cba595 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:53.014232 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a229ab-b507-4219-98cc-3c3150cba595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:53.014509 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a229ab-b507-4219-98cc-3c3150cba595 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:53.015149 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a229ab-b507-4219-98cc-3c3150cba595 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:53.017111 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a229ab-b507-4219-98cc-3c3150cba595 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:53.018833 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da91675-7cbc-4663-9048-0a73b0a419fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:53.019822 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da91675-7cbc-4663-9048-0a73b0a419fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:53.020118 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da91675-7cbc-4663-9048-0a73b0a419fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:53.020401 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da91675-7cbc-4663-9048-0a73b0a419fa admin admin] PATH_INFO: `/v3/projects/55d7243f696a44f98845e8e9ff1aaa10/users/fab792fd0f86422cb5053289d173b669/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:53.039412 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da91675-7cbc-4663-9048-0a73b0a419fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=55d7243f696a44f98845e8e9ff1aaa10, user_id=fab792fd0f86422cb5053289d173b669, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:53.042340 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da91675-7cbc-4663-9048-0a73b0a419fa admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:53.149289 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1077/2151] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:07:52 2022] POST /identity/v3/users => generated 292 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:07:53.155046 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1075/2152] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:52 2022] PUT /identity/v3/projects/55d7243f696a44f98845e8e9ff1aaa10/users/fab792fd0f86422cb5053289d173b669/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:53.156531 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a229ab-b507-4219-98cc-3c3150cba595 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:53.157166 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da91675-7cbc-4663-9048-0a73b0a419fa admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:53.169681 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a229ab-b507-4219-98cc-3c3150cba595 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:53.259828 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe8142c-8bfa-4b6a-98c6-754ee812d60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:53.260593 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8142c-8bfa-4b6a-98c6-754ee812d60c admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:53.260973 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8142c-8bfa-4b6a-98c6-754ee812d60c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:53.261343 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8142c-8bfa-4b6a-98c6-754ee812d60c admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:53.262085 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe8142c-8bfa-4b6a-98c6-754ee812d60c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:53.263460 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da91675-7cbc-4663-9048-0a73b0a419fa admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:53.264126 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe8142c-8bfa-4b6a-98c6-754ee812d60c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:53.271222 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab92435-3119-43c3-85ce-9176d4495700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:53.272336 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab92435-3119-43c3-85ce-9176d4495700 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:53.272661 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab92435-3119-43c3-85ce-9176d4495700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:53.277588 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab92435-3119-43c3-85ce-9176d4495700 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:53.277588 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab92435-3119-43c3-85ce-9176d4495700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:53.277588 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab92435-3119-43c3-85ce-9176d4495700 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:53.281359 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1078/2153] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:53 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 01:07:53.286691 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe8142c-8bfa-4b6a-98c6-754ee812d60c admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:53.288759 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1076/2154] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:53 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:53.292557 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab92435-3119-43c3-85ce-9176d4495700 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:53.301052 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe8142c-8bfa-4b6a-98c6-754ee812d60c admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:53.304157 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab92435-3119-43c3-85ce-9176d4495700 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:53.306631 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e93e73-413b-4664-8b8f-de052d752124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:53.307929 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e93e73-413b-4664-8b8f-de052d752124 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:53.307929 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e93e73-413b-4664-8b8f-de052d752124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:53.307929 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e93e73-413b-4664-8b8f-de052d752124 admin admin] PATH_INFO: `/v3/projects/389e0e44d28649d5bd12db42a8312f0e/users/1ab9ad9da6af48949165a04171a9adc2/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:53.311744 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a37ba-50e2-429b-871e-4c6ca3252994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:53.311744 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a37ba-50e2-429b-871e-4c6ca3252994 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:53.312933 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a37ba-50e2-429b-871e-4c6ca3252994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:53.313308 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a37ba-50e2-429b-871e-4c6ca3252994 admin admin] PATH_INFO: `/v3/projects/55d7243f696a44f98845e8e9ff1aaa10/users/fab792fd0f86422cb5053289d173b669/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:53.316164 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a37ba-50e2-429b-871e-4c6ca3252994 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55d7243f696a44f98845e8e9ff1aaa10, user_id=fab792fd0f86422cb5053289d173b669, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:53.320088 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a37ba-50e2-429b-871e-4c6ca3252994 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:53.334206 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:53.334852 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1077/2155] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:53 2022] PUT /identity/v3/projects/55d7243f696a44f98845e8e9ff1aaa10/users/fab792fd0f86422cb5053289d173b669/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:07:53.342085 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e93e73-413b-4664-8b8f-de052d752124 admin admin] RBAC: Authorizing `identity:create_grant(project_id=389e0e44d28649d5bd12db42a8312f0e, user_id=1ab9ad9da6af48949165a04171a9adc2, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:53.343979 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e93e73-413b-4664-8b8f-de052d752124 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:53.346615 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2990d-0883-44a3-a46c-e66d4eed9184 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:53.347753 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2990d-0883-44a3-a46c-e66d4eed9184 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:53.348750 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2990d-0883-44a3-a46c-e66d4eed9184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:53.378811 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-f9a2990d-0883-44a3-a46c-e66d4eed9184 None None] MFA Rules not processed for user `fab792fd0f86422cb5053289d173b669`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:53.380219 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f9a2990d-0883-44a3-a46c-e66d4eed9184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:53.419753 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1079/2156] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:53 2022] PUT /identity/v3/projects/389e0e44d28649d5bd12db42a8312f0e/users/1ab9ad9da6af48949165a04171a9adc2/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:07:53.421119 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e93e73-413b-4664-8b8f-de052d752124 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:53.423757 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-14e93e73-413b-4664-8b8f-de052d752124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:53.457621 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1078/2157] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:53 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:07:53.458543 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2990d-0883-44a3-a46c-e66d4eed9184 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:53.506758 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e93e73-413b-4664-8b8f-de052d752124 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:53.511746 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf09ce-17d6-4372-a61c-f61fe1803db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:53.512376 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf09ce-17d6-4372-a61c-f61fe1803db7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:53.513006 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf09ce-17d6-4372-a61c-f61fe1803db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:53.513462 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf09ce-17d6-4372-a61c-f61fe1803db7 admin admin] PATH_INFO: `/v3/users/a28ff717beef47189f1d005fe7f65a10` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:53.514409 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbf09ce-17d6-4372-a61c-f61fe1803db7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a28ff717beef47189f1d005fe7f65a10)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:53.516291 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbf09ce-17d6-4372-a61c-f61fe1803db7 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:53.537314 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2990d-0883-44a3-a46c-e66d4eed9184 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:53.543453 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5935809-e5fe-4709-8828-7730d7b20c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:53.544133 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5935809-e5fe-4709-8828-7730d7b20c70 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:53.544424 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5935809-e5fe-4709-8828-7730d7b20c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:53.544729 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5935809-e5fe-4709-8828-7730d7b20c70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:53.545370 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5935809-e5fe-4709-8828-7730d7b20c70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:53.546993 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5935809-e5fe-4709-8828-7730d7b20c70 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:53.557258 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1079/2158] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:53 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:07:53.558213 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5935809-e5fe-4709-8828-7730d7b20c70 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:53.559855 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f5935809-e5fe-4709-8828-7730d7b20c70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:53.615491 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5935809-e5fe-4709-8828-7730d7b20c70 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:53.619329 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c79889-7b04-40d8-a942-be1d773293e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:53.619968 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c79889-7b04-40d8-a942-be1d773293e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:53.620243 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c79889-7b04-40d8-a942-be1d773293e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:53.620547 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c79889-7b04-40d8-a942-be1d773293e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:53.684494 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c79889-7b04-40d8-a942-be1d773293e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:53.686003 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c79889-7b04-40d8-a942-be1d773293e2 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:53.695320 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1080/2159] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:07:53.696077 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c79889-7b04-40d8-a942-be1d773293e2 service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:53.705949 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c79889-7b04-40d8-a942-be1d773293e2 service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:53.711376 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e88802-d97f-46e3-8323-e4c477173d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:53.711979 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e88802-d97f-46e3-8323-e4c477173d8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:53.712260 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e88802-d97f-46e3-8323-e4c477173d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:53.712569 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e88802-d97f-46e3-8323-e4c477173d8f admin admin] PATH_INFO: `/v3/projects/389e0e44d28649d5bd12db42a8312f0e/users/1ab9ad9da6af48949165a04171a9adc2/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:53.714792 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e88802-d97f-46e3-8323-e4c477173d8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=389e0e44d28649d5bd12db42a8312f0e, user_id=1ab9ad9da6af48949165a04171a9adc2, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:53.717454 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e88802-d97f-46e3-8323-e4c477173d8f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:53.729837 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1081/2160] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:53 2022] PUT /identity/v3/projects/389e0e44d28649d5bd12db42a8312f0e/users/1ab9ad9da6af48949165a04171a9adc2/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:53.776333 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4bbf09ce-17d6-4372-a61c-f61fe1803db7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a28ff717beef47189f1d005fe7f65a10'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:53.777010 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4bbf09ce-17d6-4372-a61c-f61fe1803db7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a28ff717beef47189f1d005fe7f65a10'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:53.784255 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4bbf09ce-17d6-4372-a61c-f61fe1803db7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a28ff717beef47189f1d005fe7f65a10'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:53.785792 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:53.865504 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4bbf09ce-17d6-4372-a61c-f61fe1803db7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a28ff717beef47189f1d005fe7f65a10'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:53.866226 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4bbf09ce-17d6-4372-a61c-f61fe1803db7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a28ff717beef47189f1d005fe7f65a10'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:53.890372 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1080/2161] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:07:53 2022] DELETE /identity/v3/users/a28ff717beef47189f1d005fe7f65a10 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:54.253706 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:54.308788 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e88802-d97f-46e3-8323-e4c477173d8f admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:54.312583 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-99e88802-d97f-46e3-8323-e4c477173d8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:54.398983 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e88802-d97f-46e3-8323-e4c477173d8f admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:54.469195 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6298b9-d203-4713-a2d9-291b28058976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:54.469927 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6298b9-d203-4713-a2d9-291b28058976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:54.470276 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6298b9-d203-4713-a2d9-291b28058976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:54.470572 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6298b9-d203-4713-a2d9-291b28058976 admin admin] PATH_INFO: `/v3/projects/3acb006051604f59a1ddcf7a8e7145da` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:54.471710 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6298b9-d203-4713-a2d9-291b28058976 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3acb006051604f59a1ddcf7a8e7145da)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:54.473803 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6298b9-d203-4713-a2d9-291b28058976 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:54.586217 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0f6298b9-d203-4713-a2d9-291b28058976 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3acb006051604f59a1ddcf7a8e7145da', 'request_id': 'req-0f6298b9-d203-4713-a2d9-291b28058976'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:54.670667 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0f6298b9-d203-4713-a2d9-291b28058976 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3acb006051604f59a1ddcf7a8e7145da', 'request_id': 'req-0f6298b9-d203-4713-a2d9-291b28058976'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:54.677509 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0f6298b9-d203-4713-a2d9-291b28058976 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3acb006051604f59a1ddcf7a8e7145da', 'request_id': 'req-0f6298b9-d203-4713-a2d9-291b28058976'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:54.693489 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0f6298b9-d203-4713-a2d9-291b28058976 admin admin] The token cache is being invalidate because project 3acb006051604f59a1ddcf7a8e7145da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:07:54.693845 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0f6298b9-d203-4713-a2d9-291b28058976 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:54.694601 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-0f6298b9-d203-4713-a2d9-291b28058976 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:54.699736 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1082/2162] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:07:54 2022] DELETE /identity/v3/projects/3acb006051604f59a1ddcf7a8e7145da => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:54.702500 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97388c0-5c9f-40b0-a8cd-4fc0a2335471 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:54.703318 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97388c0-5c9f-40b0-a8cd-4fc0a2335471 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:54.703601 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97388c0-5c9f-40b0-a8cd-4fc0a2335471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:54.727201 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-b97388c0-5c9f-40b0-a8cd-4fc0a2335471 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:54.728436 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b97388c0-5c9f-40b0-a8cd-4fc0a2335471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:54.787099 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1081/2163] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:54.794592 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db292028-06bf-4ec1-a79b-f5cc14a36e6f None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:54.794876 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db292028-06bf-4ec1-a79b-f5cc14a36e6f None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:54.795174 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db292028-06bf-4ec1-a79b-f5cc14a36e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:54.819084 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-db292028-06bf-4ec1-a79b-f5cc14a36e6f None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:54.820374 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-db292028-06bf-4ec1-a79b-f5cc14a36e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:54.835834 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1083/2164] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:54.837351 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97388c0-5c9f-40b0-a8cd-4fc0a2335471 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:54.848203 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97388c0-5c9f-40b0-a8cd-4fc0a2335471 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:54.853928 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc4f4f-13ea-44bd-bc4b-974f97d7f939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:54.854827 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc4f4f-13ea-44bd-bc4b-974f97d7f939 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:54.855304 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc4f4f-13ea-44bd-bc4b-974f97d7f939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:54.855650 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc4f4f-13ea-44bd-bc4b-974f97d7f939 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:54.856377 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc4f4f-13ea-44bd-bc4b-974f97d7f939 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:07:54.856997 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc4f4f-13ea-44bd-bc4b-974f97d7f939 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:54.858725 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc4f4f-13ea-44bd-bc4b-974f97d7f939 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:54.872798 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1082/2165] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:07:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:07:54.873203 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:54.877046 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db292028-06bf-4ec1-a79b-f5cc14a36e6f None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:54.886623 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db292028-06bf-4ec1-a79b-f5cc14a36e6f None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:54.891901 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554aaa23-e10a-41c4-8202-c04229ffd3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:54.892604 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554aaa23-e10a-41c4-8202-c04229ffd3f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:54.892915 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554aaa23-e10a-41c4-8202-c04229ffd3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:54.893190 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554aaa23-e10a-41c4-8202-c04229ffd3f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:54.893819 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554aaa23-e10a-41c4-8202-c04229ffd3f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:54.896191 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554aaa23-e10a-41c4-8202-c04229ffd3f5 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:54.976068 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:54.983946 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1084/2166] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:07:54 2022] POST /identity/v3/projects => generated 365 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:07:54.988289 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc4f4f-13ea-44bd-bc4b-974f97d7f939 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:54.999054 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc4f4f-13ea-44bd-bc4b-974f97d7f939 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:55.060273 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5affc214-8377-438e-b8f1-7dfef49f6860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:55.060909 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affc214-8377-438e-b8f1-7dfef49f6860 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:55.061188 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affc214-8377-438e-b8f1-7dfef49f6860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:55.061462 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affc214-8377-438e-b8f1-7dfef49f6860 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:55.062108 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5affc214-8377-438e-b8f1-7dfef49f6860 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:55.064015 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5affc214-8377-438e-b8f1-7dfef49f6860 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:55.168520 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1083/2167] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:07:54 2022] POST /identity/v3/users => generated 283 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:07:55.172325 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554aaa23-e10a-41c4-8202-c04229ffd3f5 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:55.183747 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554aaa23-e10a-41c4-8202-c04229ffd3f5 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:55.188503 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db54c4f-de0c-4d68-b2bb-7852bd78f8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:55.189584 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db54c4f-de0c-4d68-b2bb-7852bd78f8af admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:55.189990 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db54c4f-de0c-4d68-b2bb-7852bd78f8af admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:55.190271 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db54c4f-de0c-4d68-b2bb-7852bd78f8af admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:55.190929 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db54c4f-de0c-4d68-b2bb-7852bd78f8af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:55.192760 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db54c4f-de0c-4d68-b2bb-7852bd78f8af admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:55.205632 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0a287-f873-4ddf-8bd4-ed399ce00c79 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:55.205921 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0a287-f873-4ddf-8bd4-ed399ce00c79 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:55.206197 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0a287-f873-4ddf-8bd4-ed399ce00c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:55.212363 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1085/2168] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:55 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 01:07:55.216512 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db54c4f-de0c-4d68-b2bb-7852bd78f8af admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:55.235770 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db54c4f-de0c-4d68-b2bb-7852bd78f8af admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:55.242623 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b575b-f358-4f11-8781-50198a28805e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:55.243239 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b575b-f358-4f11-8781-50198a28805e admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:55.243519 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b575b-f358-4f11-8781-50198a28805e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:55.243893 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b575b-f358-4f11-8781-50198a28805e admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:55.244706 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b575b-f358-4f11-8781-50198a28805e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:55.246313 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b575b-f358-4f11-8781-50198a28805e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:55.260059 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1086/2169] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:07:55 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 01:07:55.264225 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-a4a0a287-f873-4ddf-8bd4-ed399ce00c79 None None] MFA Rules not processed for user `1ab9ad9da6af48949165a04171a9adc2`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:55.266010 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b575b-f358-4f11-8781-50198a28805e admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:55.266384 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a4a0a287-f873-4ddf-8bd4-ed399ce00c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:55.277528 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b575b-f358-4f11-8781-50198a28805e admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:55.284452 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73410c06-079e-476b-a3cb-bcab8bf1df98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:55.286026 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73410c06-079e-476b-a3cb-bcab8bf1df98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:55.286449 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73410c06-079e-476b-a3cb-bcab8bf1df98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:55.286805 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73410c06-079e-476b-a3cb-bcab8bf1df98 admin admin] PATH_INFO: `/v3/projects/f31b99bf5a9a48c0a3901675ae796e15/users/bdaf151db2884612af5039fabd124431/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:55.305794 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73410c06-079e-476b-a3cb-bcab8bf1df98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f31b99bf5a9a48c0a3901675ae796e15, user_id=bdaf151db2884612af5039fabd124431, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:55.309230 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73410c06-079e-476b-a3cb-bcab8bf1df98 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:55.369804 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1084/2170] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:55 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:07:55.381472 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6121e4aa-1174-49ef-9d85-22256dc17dfe None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:55.381818 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6121e4aa-1174-49ef-9d85-22256dc17dfe None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:55.382099 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6121e4aa-1174-49ef-9d85-22256dc17dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:55.401601 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-6121e4aa-1174-49ef-9d85-22256dc17dfe None None] MFA Rules not processed for user `7a1fc708e8cd4b42baa8a661a625606b`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:55.402995 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6121e4aa-1174-49ef-9d85-22256dc17dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:55.411859 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1087/2171] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:07:55 2022] PUT /identity/v3/projects/f31b99bf5a9a48c0a3901675ae796e15/users/bdaf151db2884612af5039fabd124431/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:07:55.422073 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d26ca7-7ee8-48c5-9ffa-df4697480e1a None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:55.422362 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d26ca7-7ee8-48c5-9ffa-df4697480e1a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:55.422641 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d26ca7-7ee8-48c5-9ffa-df4697480e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:55.452601 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-65d26ca7-7ee8-48c5-9ffa-df4697480e1a None None] MFA Rules not processed for user `bdaf151db2884612af5039fabd124431`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:55.453943 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-65d26ca7-7ee8-48c5-9ffa-df4697480e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:55.462284 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1085/2172] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:55.474862 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6121e4aa-1174-49ef-9d85-22256dc17dfe None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:55.476150 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6121e4aa-1174-49ef-9d85-22256dc17dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:55.521187 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1088/2173] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:07:55.532404 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d26ca7-7ee8-48c5-9ffa-df4697480e1a None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:55.534246 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-65d26ca7-7ee8-48c5-9ffa-df4697480e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:55.557569 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6121e4aa-1174-49ef-9d85-22256dc17dfe None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:55.563630 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d736a9-0a25-4314-ad85-2c1758eb04f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:55.564239 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d736a9-0a25-4314-ad85-2c1758eb04f4 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:55.564547 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d736a9-0a25-4314-ad85-2c1758eb04f4 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:55.564823 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d736a9-0a25-4314-ad85-2c1758eb04f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:55.566668 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-24d736a9-0a25-4314-ad85-2c1758eb04f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:55.577422 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d736a9-0a25-4314-ad85-2c1758eb04f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:55.579020 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d736a9-0a25-4314-ad85-2c1758eb04f4 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:55.588416 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1086/2174] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 01:07:55.592945 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d26ca7-7ee8-48c5-9ffa-df4697480e1a None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:55.596687 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce0102-a4af-4c00-9ff3-420fd511aad4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:55.597293 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce0102-a4af-4c00-9ff3-420fd511aad4 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:55.597571 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce0102-a4af-4c00-9ff3-420fd511aad4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:55.598527 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce0102-a4af-4c00-9ff3-420fd511aad4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:55.607561 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce0102-a4af-4c00-9ff3-420fd511aad4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:55.609117 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce0102-a4af-4c00-9ff3-420fd511aad4 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:55.618417 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1089/2175] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 01:07:55.825002 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:55.967835 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:56.625092 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:56.710759 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:56.719692 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d736a9-0a25-4314-ad85-2c1758eb04f4 service nova] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:56.720705 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-24d736a9-0a25-4314-ad85-2c1758eb04f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:56.732667 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d736a9-0a25-4314-ad85-2c1758eb04f4 service nova] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:56.736285 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d7eac3-3a03-4cc9-a344-94366a0ba784 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:56.737006 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7eac3-3a03-4cc9-a344-94366a0ba784 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:56.737358 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7eac3-3a03-4cc9-a344-94366a0ba784 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:56.737638 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7eac3-3a03-4cc9-a344-94366a0ba784 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:56.739355 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d8d7eac3-3a03-4cc9-a344-94366a0ba784 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:56.802841 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d7eac3-3a03-4cc9-a344-94366a0ba784 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:56.804332 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d7eac3-3a03-4cc9-a344-94366a0ba784 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:56.827218 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1087/2176] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:07:57.851909 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce0102-a4af-4c00-9ff3-420fd511aad4 service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:57.853347 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-50ce0102-a4af-4c00-9ff3-420fd511aad4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:57.921542 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce0102-a4af-4c00-9ff3-420fd511aad4 service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:57.926505 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c520be5-aa0d-49b9-a2e5-cf07668f7c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:57.931752 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c520be5-aa0d-49b9-a2e5-cf07668f7c88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:57.931752 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c520be5-aa0d-49b9-a2e5-cf07668f7c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:57.931752 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c520be5-aa0d-49b9-a2e5-cf07668f7c88 admin admin] PATH_INFO: `/v3/users/f2acc3fe07a24687bc768d86e21d8de1` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:57.931752 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c520be5-aa0d-49b9-a2e5-cf07668f7c88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2acc3fe07a24687bc768d86e21d8de1)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:57.931752 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c520be5-aa0d-49b9-a2e5-cf07668f7c88 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:57.979095 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:58.177083 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-7c520be5-aa0d-49b9-a2e5-cf07668f7c88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2acc3fe07a24687bc768d86e21d8de1'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:58.177802 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-7c520be5-aa0d-49b9-a2e5-cf07668f7c88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2acc3fe07a24687bc768d86e21d8de1'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:58.253127 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-7c520be5-aa0d-49b9-a2e5-cf07668f7c88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2acc3fe07a24687bc768d86e21d8de1'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:58.254531 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-7c520be5-aa0d-49b9-a2e5-cf07668f7c88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2acc3fe07a24687bc768d86e21d8de1'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:58.267764 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-7c520be5-aa0d-49b9-a2e5-cf07668f7c88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2acc3fe07a24687bc768d86e21d8de1'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:58.282956 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:58.284587 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1090/2177] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:07:57 2022] DELETE /identity/v3/users/f2acc3fe07a24687bc768d86e21d8de1 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:58.286474 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb62649-41c8-48d8-8589-8b32c9642ac0 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:58.286823 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb62649-41c8-48d8-8589-8b32c9642ac0 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:58.287107 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb62649-41c8-48d8-8589-8b32c9642ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:58.310081 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-acb62649-41c8-48d8-8589-8b32c9642ac0 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:07:58.311407 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-acb62649-41c8-48d8-8589-8b32c9642ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:58.373963 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1088/2178] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:07:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:07:58.384523 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c520be5-aa0d-49b9-a2e5-cf07668f7c88 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:58.389590 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-7c520be5-aa0d-49b9-a2e5-cf07668f7c88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:58.468940 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c520be5-aa0d-49b9-a2e5-cf07668f7c88 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:58.475024 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c3f76-2053-4091-b5b3-fc61a7683731 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:58.475024 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c3f76-2053-4091-b5b3-fc61a7683731 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:58.475024 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c3f76-2053-4091-b5b3-fc61a7683731 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:58.475024 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c3f76-2053-4091-b5b3-fc61a7683731 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:58.485648 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7c3f76-2053-4091-b5b3-fc61a7683731 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:58.487282 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7c3f76-2053-4091-b5b3-fc61a7683731 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:58.498584 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1091/2179] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:07:58.609081 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb62649-41c8-48d8-8589-8b32c9642ac0 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:58.615127 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-acb62649-41c8-48d8-8589-8b32c9642ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:58.681657 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb62649-41c8-48d8-8589-8b32c9642ac0 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:58.689356 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7fe202-958a-46cc-aa11-ee6a8133dbee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:58.689977 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fe202-958a-46cc-aa11-ee6a8133dbee service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:58.690255 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fe202-958a-46cc-aa11-ee6a8133dbee service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:58.690527 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fe202-958a-46cc-aa11-ee6a8133dbee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:58.693918 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ad7fe202-958a-46cc-aa11-ee6a8133dbee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:58.755142 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:58.771375 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7fe202-958a-46cc-aa11-ee6a8133dbee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:58.772875 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7fe202-958a-46cc-aa11-ee6a8133dbee service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:58.782745 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1089/2180] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:07:58 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:07:59.068311 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c3f76-2053-4091-b5b3-fc61a7683731 service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:59.070761 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3e7c3f76-2053-4091-b5b3-fc61a7683731 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:07:59.087038 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c3f76-2053-4091-b5b3-fc61a7683731 service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:59.092475 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db63b2b7-d8f6-4c9a-a622-f9f713a93cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:59.093499 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63b2b7-d8f6-4c9a-a622-f9f713a93cfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:59.093935 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63b2b7-d8f6-4c9a-a622-f9f713a93cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:59.094564 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63b2b7-d8f6-4c9a-a622-f9f713a93cfd admin admin] PATH_INFO: `/v3/projects/f6e9daaec12642908434416f4c275b29` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:59.095661 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db63b2b7-d8f6-4c9a-a622-f9f713a93cfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6e9daaec12642908434416f4c275b29)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:59.099229 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db63b2b7-d8f6-4c9a-a622-f9f713a93cfd admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:59.237464 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-db63b2b7-d8f6-4c9a-a622-f9f713a93cfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6e9daaec12642908434416f4c275b29', 'request_id': 'req-db63b2b7-d8f6-4c9a-a622-f9f713a93cfd'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:59.340926 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-db63b2b7-d8f6-4c9a-a622-f9f713a93cfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6e9daaec12642908434416f4c275b29', 'request_id': 'req-db63b2b7-d8f6-4c9a-a622-f9f713a93cfd'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:59.346935 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-db63b2b7-d8f6-4c9a-a622-f9f713a93cfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6e9daaec12642908434416f4c275b29', 'request_id': 'req-db63b2b7-d8f6-4c9a-a622-f9f713a93cfd'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:59.358866 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-db63b2b7-d8f6-4c9a-a622-f9f713a93cfd admin admin] The token cache is being invalidate because project f6e9daaec12642908434416f4c275b29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:07:59.359139 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-db63b2b7-d8f6-4c9a-a622-f9f713a93cfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:59.359777 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-db63b2b7-d8f6-4c9a-a622-f9f713a93cfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:59.361304 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1092/2181] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:07:59 2022] DELETE /identity/v3/projects/f6e9daaec12642908434416f4c275b29 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:59.365677 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7fe202-958a-46cc-aa11-ee6a8133dbee service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:07:59.370126 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:59.377332 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7fe202-958a-46cc-aa11-ee6a8133dbee service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:07:59.445854 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b58dea0-81ec-46fc-92ad-37537664c570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:07:59.446544 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58dea0-81ec-46fc-92ad-37537664c570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:07:59.446828 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58dea0-81ec-46fc-92ad-37537664c570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:07:59.447198 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58dea0-81ec-46fc-92ad-37537664c570 admin admin] PATH_INFO: `/v3/users/55f2f29844374152bc1ebd338074cd18` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:07:59.448544 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b58dea0-81ec-46fc-92ad-37537664c570 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55f2f29844374152bc1ebd338074cd18)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:07:59.450527 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b58dea0-81ec-46fc-92ad-37537664c570 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:07:59.658538 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6b58dea0-81ec-46fc-92ad-37537664c570 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55f2f29844374152bc1ebd338074cd18'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:59.659240 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6b58dea0-81ec-46fc-92ad-37537664c570 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55f2f29844374152bc1ebd338074cd18'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:59.667181 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6b58dea0-81ec-46fc-92ad-37537664c570 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55f2f29844374152bc1ebd338074cd18'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:59.771697 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:59.772375 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6b58dea0-81ec-46fc-92ad-37537664c570 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55f2f29844374152bc1ebd338074cd18'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:59.773116 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6b58dea0-81ec-46fc-92ad-37537664c570 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55f2f29844374152bc1ebd338074cd18'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:07:59.791656 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1090/2182] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:07:59 2022] DELETE /identity/v3/users/55f2f29844374152bc1ebd338074cd18 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:07:59.981151 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:00.393031 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db63b2b7-d8f6-4c9a-a622-f9f713a93cfd admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:00.394942 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-db63b2b7-d8f6-4c9a-a622-f9f713a93cfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:00.408991 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db63b2b7-d8f6-4c9a-a622-f9f713a93cfd admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:00.472444 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c47685-576f-4b0e-9bb8-ed6165cba88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:00.473630 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c47685-576f-4b0e-9bb8-ed6165cba88f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:00.473917 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c47685-576f-4b0e-9bb8-ed6165cba88f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:00.474198 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c47685-576f-4b0e-9bb8-ed6165cba88f admin admin] PATH_INFO: `/v3/projects/4aa21864731646f0b73ac2d877c9aadc` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:00.475510 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c47685-576f-4b0e-9bb8-ed6165cba88f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4aa21864731646f0b73ac2d877c9aadc)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:00.477775 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c47685-576f-4b0e-9bb8-ed6165cba88f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:00.596679 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:00.665815 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-53c47685-576f-4b0e-9bb8-ed6165cba88f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4aa21864731646f0b73ac2d877c9aadc', 'request_id': 'req-53c47685-576f-4b0e-9bb8-ed6165cba88f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:00.770670 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-53c47685-576f-4b0e-9bb8-ed6165cba88f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4aa21864731646f0b73ac2d877c9aadc', 'request_id': 'req-53c47685-576f-4b0e-9bb8-ed6165cba88f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:00.771994 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:00.780238 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-53c47685-576f-4b0e-9bb8-ed6165cba88f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4aa21864731646f0b73ac2d877c9aadc', 'request_id': 'req-53c47685-576f-4b0e-9bb8-ed6165cba88f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:00.794153 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-53c47685-576f-4b0e-9bb8-ed6165cba88f admin admin] The token cache is being invalidate because project 4aa21864731646f0b73ac2d877c9aadc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:08:00.794992 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-53c47685-576f-4b0e-9bb8-ed6165cba88f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:00.796662 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-53c47685-576f-4b0e-9bb8-ed6165cba88f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:00.802092 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1093/2183] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:08:00 2022] DELETE /identity/v3/projects/4aa21864731646f0b73ac2d877c9aadc => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:00.804189 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77a2ff-7cdb-4562-afc5-0109c6d3bfe9 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:00.804730 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77a2ff-7cdb-4562-afc5-0109c6d3bfe9 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:00.807811 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77a2ff-7cdb-4562-afc5-0109c6d3bfe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:00.834718 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-ae77a2ff-7cdb-4562-afc5-0109c6d3bfe9 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:00.836203 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ae77a2ff-7cdb-4562-afc5-0109c6d3bfe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:00.907406 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1091/2184] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:00.915545 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab891f6-bcae-44c2-a400-769f207e4ba2 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:00.915922 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab891f6-bcae-44c2-a400-769f207e4ba2 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:00.916388 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab891f6-bcae-44c2-a400-769f207e4ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:00.936820 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-3ab891f6-bcae-44c2-a400-769f207e4ba2 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:00.938503 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3ab891f6-bcae-44c2-a400-769f207e4ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:00.947273 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1094/2185] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:08:00.951541 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae77a2ff-7cdb-4562-afc5-0109c6d3bfe9 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:00.961645 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae77a2ff-7cdb-4562-afc5-0109c6d3bfe9 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:00.967593 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f7ad87-c676-44dd-82d1-5e1136f0d9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:00.967593 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7ad87-c676-44dd-82d1-5e1136f0d9cf admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:00.967593 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7ad87-c676-44dd-82d1-5e1136f0d9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:00.967593 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7ad87-c676-44dd-82d1-5e1136f0d9cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:00.968621 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f7ad87-c676-44dd-82d1-5e1136f0d9cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:08:00.968621 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f7ad87-c676-44dd-82d1-5e1136f0d9cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:00.970213 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f7ad87-c676-44dd-82d1-5e1136f0d9cf admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:00.985902 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1092/2186] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:08:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:08:00.987133 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab891f6-bcae-44c2-a400-769f207e4ba2 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:00.997990 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab891f6-bcae-44c2-a400-769f207e4ba2 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:01.002817 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0297c-7f08-41e0-bc42-db267cae678b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:01.003422 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0297c-7f08-41e0-bc42-db267cae678b admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:01.003700 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0297c-7f08-41e0-bc42-db267cae678b admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:01.004740 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0297c-7f08-41e0-bc42-db267cae678b admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:01.005430 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af0297c-7f08-41e0-bc42-db267cae678b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:01.007375 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af0297c-7f08-41e0-bc42-db267cae678b admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:01.103431 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1095/2187] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:08:00 2022] POST /identity/v3/projects => generated 409 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:08:01.107505 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f7ad87-c676-44dd-82d1-5e1136f0d9cf admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:01.118160 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f7ad87-c676-44dd-82d1-5e1136f0d9cf admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:01.188366 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13d40b-2a12-47f5-923b-090042df7971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:01.195840 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13d40b-2a12-47f5-923b-090042df7971 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:01.195840 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13d40b-2a12-47f5-923b-090042df7971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:01.195840 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13d40b-2a12-47f5-923b-090042df7971 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:01.195840 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13d40b-2a12-47f5-923b-090042df7971 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:01.195840 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13d40b-2a12-47f5-923b-090042df7971 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:01.294128 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1093/2188] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:08:01 2022] POST /identity/v3/users => generated 305 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:08:01.298327 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0297c-7f08-41e0-bc42-db267cae678b admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:01.309869 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0297c-7f08-41e0-bc42-db267cae678b admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:01.314741 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac3437-fabd-44e6-90f5-7a31f6b97b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:01.315361 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac3437-fabd-44e6-90f5-7a31f6b97b25 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:01.315633 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac3437-fabd-44e6-90f5-7a31f6b97b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:01.315914 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac3437-fabd-44e6-90f5-7a31f6b97b25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:01.317201 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdac3437-fabd-44e6-90f5-7a31f6b97b25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:01.318813 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdac3437-fabd-44e6-90f5-7a31f6b97b25 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:01.328875 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1096/2189] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:08:01.333533 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13d40b-2a12-47f5-923b-090042df7971 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:01.343973 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13d40b-2a12-47f5-923b-090042df7971 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:01.350052 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819aa05e-ef33-4a3b-bf86-3fa7efd9fd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:01.350696 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819aa05e-ef33-4a3b-bf86-3fa7efd9fd04 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:01.350968 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819aa05e-ef33-4a3b-bf86-3fa7efd9fd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:01.351245 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819aa05e-ef33-4a3b-bf86-3fa7efd9fd04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:01.351973 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819aa05e-ef33-4a3b-bf86-3fa7efd9fd04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:01.366135 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819aa05e-ef33-4a3b-bf86-3fa7efd9fd04 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:01.366135 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1094/2190] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:08:01.369099 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac3437-fabd-44e6-90f5-7a31f6b97b25 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:01.385932 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac3437-fabd-44e6-90f5-7a31f6b97b25 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:01.391256 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bad89b7-472f-4ab7-9ef7-62613a1055ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:01.395759 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad89b7-472f-4ab7-9ef7-62613a1055ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:01.395759 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad89b7-472f-4ab7-9ef7-62613a1055ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:01.395759 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad89b7-472f-4ab7-9ef7-62613a1055ca admin admin] PATH_INFO: `/v3/projects/ba08a6957ed346d0b747835e3dc15696/users/42063d51ae8f4afea4a0426cd7339903/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:01.408964 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bad89b7-472f-4ab7-9ef7-62613a1055ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba08a6957ed346d0b747835e3dc15696, user_id=42063d51ae8f4afea4a0426cd7339903, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:01.411763 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bad89b7-472f-4ab7-9ef7-62613a1055ca admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:01.496738 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1097/2191] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:08:01 2022] PUT /identity/v3/projects/ba08a6957ed346d0b747835e3dc15696/users/42063d51ae8f4afea4a0426cd7339903/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:01.508526 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbdabf-78b7-447b-aafb-e794b0a9ed54 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:01.509040 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbdabf-78b7-447b-aafb-e794b0a9ed54 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:01.509458 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbdabf-78b7-447b-aafb-e794b0a9ed54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:01.539111 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-a6bbdabf-78b7-447b-aafb-e794b0a9ed54 None None] MFA Rules not processed for user `42063d51ae8f4afea4a0426cd7339903`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:01.540335 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a6bbdabf-78b7-447b-aafb-e794b0a9ed54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:01.610029 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1095/2192] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:01 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:08:01.615377 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bad89b7-472f-4ab7-9ef7-62613a1055ca admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:01.682683 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bad89b7-472f-4ab7-9ef7-62613a1055ca admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:01.683866 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e1979-3256-4d59-a4b0-951f35b2d344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:01.684779 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e1979-3256-4d59-a4b0-951f35b2d344 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:01.684779 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e1979-3256-4d59-a4b0-951f35b2d344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:01.685069 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e1979-3256-4d59-a4b0-951f35b2d344 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:01.685684 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37e1979-3256-4d59-a4b0-951f35b2d344 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:01.687587 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37e1979-3256-4d59-a4b0-951f35b2d344 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:01.804584 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:01.812334 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1098/2193] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:08:01 2022] POST /identity/v3/projects => generated 409 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:08:01.817116 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bbdabf-78b7-447b-aafb-e794b0a9ed54 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:01.828135 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bbdabf-78b7-447b-aafb-e794b0a9ed54 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:01.891609 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18aec6c-d818-45a7-ada1-e9a71ec7a3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:01.892358 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18aec6c-d818-45a7-ada1-e9a71ec7a3f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:01.892696 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18aec6c-d818-45a7-ada1-e9a71ec7a3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:01.893020 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18aec6c-d818-45a7-ada1-e9a71ec7a3f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:01.893662 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18aec6c-d818-45a7-ada1-e9a71ec7a3f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:01.895845 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18aec6c-d818-45a7-ada1-e9a71ec7a3f1 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:01.999199 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:02.042561 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1096/2194] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:08:01 2022] POST /identity/v3/users => generated 311 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:08:02.046711 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e1979-3256-4d59-a4b0-951f35b2d344 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:02.078347 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e1979-3256-4d59-a4b0-951f35b2d344 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:02.084394 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25dc4e3-5e55-4532-86ef-eaadef0b6155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:02.089664 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25dc4e3-5e55-4532-86ef-eaadef0b6155 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:02.089664 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25dc4e3-5e55-4532-86ef-eaadef0b6155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:02.089664 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25dc4e3-5e55-4532-86ef-eaadef0b6155 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:02.089664 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25dc4e3-5e55-4532-86ef-eaadef0b6155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:02.089664 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25dc4e3-5e55-4532-86ef-eaadef0b6155 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:02.109277 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1099/2195] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:08:02.117529 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18aec6c-d818-45a7-ada1-e9a71ec7a3f1 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:02.138976 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18aec6c-d818-45a7-ada1-e9a71ec7a3f1 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:02.151958 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a324b3a-24eb-48e2-b3b5-b441d5f21f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:02.155174 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a324b3a-24eb-48e2-b3b5-b441d5f21f5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:02.155477 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a324b3a-24eb-48e2-b3b5-b441d5f21f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:02.155762 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a324b3a-24eb-48e2-b3b5-b441d5f21f5a admin admin] PATH_INFO: `/v3/projects/5d7be4e7725d4f1fa9c10fc1d26a70b2/users/05979affd0924ba7a7583171e05efc60/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:02.175026 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a324b3a-24eb-48e2-b3b5-b441d5f21f5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d7be4e7725d4f1fa9c10fc1d26a70b2, user_id=05979affd0924ba7a7583171e05efc60, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:02.179369 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a324b3a-24eb-48e2-b3b5-b441d5f21f5a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:02.259536 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1097/2196] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:08:02 2022] PUT /identity/v3/projects/5d7be4e7725d4f1fa9c10fc1d26a70b2/users/05979affd0924ba7a7583171e05efc60/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:08:02.264048 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25dc4e3-5e55-4532-86ef-eaadef0b6155 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:02.281119 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25dc4e3-5e55-4532-86ef-eaadef0b6155 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:02.365838 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760fb95f-9198-416c-9a37-8678c2624d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:02.366460 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760fb95f-9198-416c-9a37-8678c2624d25 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:02.366794 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760fb95f-9198-416c-9a37-8678c2624d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:02.367067 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760fb95f-9198-416c-9a37-8678c2624d25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:02.367712 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760fb95f-9198-416c-9a37-8678c2624d25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:02.370154 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760fb95f-9198-416c-9a37-8678c2624d25 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:02.381503 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1100/2197] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:08:02.385637 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a324b3a-24eb-48e2-b3b5-b441d5f21f5a admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:02.398727 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a324b3a-24eb-48e2-b3b5-b441d5f21f5a admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:02.404298 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e2a35-8e09-4484-aed1-bf2ea48290fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:02.405282 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e2a35-8e09-4484-aed1-bf2ea48290fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:02.405628 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e2a35-8e09-4484-aed1-bf2ea48290fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:02.406017 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e2a35-8e09-4484-aed1-bf2ea48290fa admin admin] PATH_INFO: `/v3/projects/5d7be4e7725d4f1fa9c10fc1d26a70b2/users/05979affd0924ba7a7583171e05efc60/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:02.408211 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022e2a35-8e09-4484-aed1-bf2ea48290fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d7be4e7725d4f1fa9c10fc1d26a70b2, user_id=05979affd0924ba7a7583171e05efc60, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:02.411606 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022e2a35-8e09-4484-aed1-bf2ea48290fa admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:02.423455 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1098/2198] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:08:02 2022] PUT /identity/v3/projects/5d7be4e7725d4f1fa9c10fc1d26a70b2/users/05979affd0924ba7a7583171e05efc60/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:02.439718 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588c169-6433-414e-80d9-a58a454ecb5b None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:02.440186 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588c169-6433-414e-80d9-a58a454ecb5b None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:02.440186 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588c169-6433-414e-80d9-a58a454ecb5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:02.468613 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-8588c169-6433-414e-80d9-a58a454ecb5b None None] MFA Rules not processed for user `05979affd0924ba7a7583171e05efc60`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:02.469817 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8588c169-6433-414e-80d9-a58a454ecb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:02.539714 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1101/2199] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:02 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:02.550631 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e2a35-8e09-4484-aed1-bf2ea48290fa admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:02.552326 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-022e2a35-8e09-4484-aed1-bf2ea48290fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:02.614483 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e2a35-8e09-4484-aed1-bf2ea48290fa admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:02.618296 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f13fe-3564-472c-b441-a1d31d886694 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:02.618994 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f13fe-3564-472c-b441-a1d31d886694 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:02.619274 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f13fe-3564-472c-b441-a1d31d886694 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:02.619545 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f13fe-3564-472c-b441-a1d31d886694 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:02.628788 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f13fe-3564-472c-b441-a1d31d886694 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:02.630216 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f13fe-3564-472c-b441-a1d31d886694 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:02.641616 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1099/2200] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:08:02 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:08:02.807414 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:02.978486 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8588c169-6433-414e-80d9-a58a454ecb5b None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:02.988211 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8588c169-6433-414e-80d9-a58a454ecb5b None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:02.992248 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876616c-8294-4c66-9073-88aa78de92ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:02.992881 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876616c-8294-4c66-9073-88aa78de92ac service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:02.993162 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876616c-8294-4c66-9073-88aa78de92ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:02.993437 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876616c-8294-4c66-9073-88aa78de92ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:03.050054 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8876616c-8294-4c66-9073-88aa78de92ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:03.052188 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8876616c-8294-4c66-9073-88aa78de92ac service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:03.054091 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:03.063998 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1102/2201] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:08:02 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:08:03.714740 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f13fe-3564-472c-b441-a1d31d886694 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:03.716249 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a74f13fe-3564-472c-b441-a1d31d886694 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:03.730506 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:03.767500 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:03.792706 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f13fe-3564-472c-b441-a1d31d886694 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:03.793484 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba99fa-b59b-4e47-8f3c-8fd092661ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:03.794115 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba99fa-b59b-4e47-8f3c-8fd092661ca2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:03.794432 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba99fa-b59b-4e47-8f3c-8fd092661ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:03.794726 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba99fa-b59b-4e47-8f3c-8fd092661ca2 admin admin] PATH_INFO: `/v3/users/fc3983ab83e24d3784e79fd501956191` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:03.795887 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ba99fa-b59b-4e47-8f3c-8fd092661ca2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc3983ab83e24d3784e79fd501956191)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:03.797822 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ba99fa-b59b-4e47-8f3c-8fd092661ca2 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:03.983095 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e9ba99fa-b59b-4e47-8f3c-8fd092661ca2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc3983ab83e24d3784e79fd501956191'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:03.983757 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e9ba99fa-b59b-4e47-8f3c-8fd092661ca2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc3983ab83e24d3784e79fd501956191'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:03.990574 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e9ba99fa-b59b-4e47-8f3c-8fd092661ca2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc3983ab83e24d3784e79fd501956191'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:04.070358 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:04.071624 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e9ba99fa-b59b-4e47-8f3c-8fd092661ca2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc3983ab83e24d3784e79fd501956191'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:04.072323 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e9ba99fa-b59b-4e47-8f3c-8fd092661ca2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc3983ab83e24d3784e79fd501956191'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:04.090863 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1100/2202] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:08:03 2022] DELETE /identity/v3/users/fc3983ab83e24d3784e79fd501956191 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:08:04.095643 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f898a-da60-44dc-8047-c4707ecd66ce None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:04.095932 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f898a-da60-44dc-8047-c4707ecd66ce None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:04.103142 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f898a-da60-44dc-8047-c4707ecd66ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:04.116319 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-b77f898a-da60-44dc-8047-c4707ecd66ce None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:04.117561 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b77f898a-da60-44dc-8047-c4707ecd66ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:04.149784 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:04.180678 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1103/2203] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:04.190511 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba99fa-b59b-4e47-8f3c-8fd092661ca2 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:04.192609 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e9ba99fa-b59b-4e47-8f3c-8fd092661ca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:04.252683 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba99fa-b59b-4e47-8f3c-8fd092661ca2 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:04.256989 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8042e9-14ed-4328-8dc8-00d250f71653 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:04.257581 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8042e9-14ed-4328-8dc8-00d250f71653 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:04.257858 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8042e9-14ed-4328-8dc8-00d250f71653 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:04.258130 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8042e9-14ed-4328-8dc8-00d250f71653 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:04.263058 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:04.267676 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8042e9-14ed-4328-8dc8-00d250f71653 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:04.269155 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8042e9-14ed-4328-8dc8-00d250f71653 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:04.280013 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1101/2204] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:08:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 01:08:04.289722 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:04.530613 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:04.587297 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:04.690577 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f898a-da60-44dc-8047-c4707ecd66ce None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:04.692690 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b77f898a-da60-44dc-8047-c4707ecd66ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:04.705518 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f898a-da60-44dc-8047-c4707ecd66ce None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:04.710970 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9147f-0d56-48eb-80fb-71ed0f609f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:04.711797 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9147f-0d56-48eb-80fb-71ed0f609f60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:04.712159 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9147f-0d56-48eb-80fb-71ed0f609f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:04.712575 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9147f-0d56-48eb-80fb-71ed0f609f60 admin admin] PATH_INFO: `/v3/projects/ceded6b844164be194c5d6efbb1d3b90` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:04.720419 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9147f-0d56-48eb-80fb-71ed0f609f60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ceded6b844164be194c5d6efbb1d3b90)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:04.720419 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9147f-0d56-48eb-80fb-71ed0f609f60 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:04.810178 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:04.868217 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a8e9147f-0d56-48eb-80fb-71ed0f609f60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ceded6b844164be194c5d6efbb1d3b90', 'request_id': 'req-a8e9147f-0d56-48eb-80fb-71ed0f609f60'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:04.941564 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a8e9147f-0d56-48eb-80fb-71ed0f609f60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ceded6b844164be194c5d6efbb1d3b90', 'request_id': 'req-a8e9147f-0d56-48eb-80fb-71ed0f609f60'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:04.948577 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a8e9147f-0d56-48eb-80fb-71ed0f609f60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ceded6b844164be194c5d6efbb1d3b90', 'request_id': 'req-a8e9147f-0d56-48eb-80fb-71ed0f609f60'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:04.966355 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a8e9147f-0d56-48eb-80fb-71ed0f609f60 admin admin] The token cache is being invalidate because project ceded6b844164be194c5d6efbb1d3b90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:08:04.966355 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a8e9147f-0d56-48eb-80fb-71ed0f609f60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:04.966355 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a8e9147f-0d56-48eb-80fb-71ed0f609f60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:04.969409 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1104/2205] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:08:04 2022] DELETE /identity/v3/projects/ceded6b844164be194c5d6efbb1d3b90 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:04.970358 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8042e9-14ed-4328-8dc8-00d250f71653 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:04.981425 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8042e9-14ed-4328-8dc8-00d250f71653 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:05.038446 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b413b-1c7a-4ae2-970f-eba76b6197a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:05.039068 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b413b-1c7a-4ae2-970f-eba76b6197a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:05.039357 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b413b-1c7a-4ae2-970f-eba76b6197a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:05.039619 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b413b-1c7a-4ae2-970f-eba76b6197a7 admin admin] PATH_INFO: `/v3/users/ab7fddc495c04c3084add4ede84fbaea` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:05.040751 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b413b-1c7a-4ae2-970f-eba76b6197a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab7fddc495c04c3084add4ede84fbaea)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:05.042941 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b413b-1c7a-4ae2-970f-eba76b6197a7 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:05.055795 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:05.212886 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-156b413b-1c7a-4ae2-970f-eba76b6197a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab7fddc495c04c3084add4ede84fbaea'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:05.213756 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-156b413b-1c7a-4ae2-970f-eba76b6197a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab7fddc495c04c3084add4ede84fbaea'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:05.221588 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-156b413b-1c7a-4ae2-970f-eba76b6197a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab7fddc495c04c3084add4ede84fbaea'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:05.316029 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-156b413b-1c7a-4ae2-970f-eba76b6197a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab7fddc495c04c3084add4ede84fbaea'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:05.316301 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:05.317733 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-156b413b-1c7a-4ae2-970f-eba76b6197a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab7fddc495c04c3084add4ede84fbaea'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:05.335474 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1102/2206] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:08:04 2022] DELETE /identity/v3/users/ab7fddc495c04c3084add4ede84fbaea => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:05.535442 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:05.737858 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9147f-0d56-48eb-80fb-71ed0f609f60 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:05.739989 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a8e9147f-0d56-48eb-80fb-71ed0f609f60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:05.753612 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9147f-0d56-48eb-80fb-71ed0f609f60 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:05.827007 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7934aa-9977-4705-b04a-1d736873fb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:05.828270 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7934aa-9977-4705-b04a-1d736873fb95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:05.828752 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7934aa-9977-4705-b04a-1d736873fb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:05.829169 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7934aa-9977-4705-b04a-1d736873fb95 admin admin] PATH_INFO: `/v3/projects/53a25412091c4285a3124433e04f4464` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:05.830510 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7934aa-9977-4705-b04a-1d736873fb95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53a25412091c4285a3124433e04f4464)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:05.832700 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7934aa-9977-4705-b04a-1d736873fb95 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:05.968448 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ed7934aa-9977-4705-b04a-1d736873fb95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53a25412091c4285a3124433e04f4464', 'request_id': 'req-ed7934aa-9977-4705-b04a-1d736873fb95'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:05.974661 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:06.064540 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ed7934aa-9977-4705-b04a-1d736873fb95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53a25412091c4285a3124433e04f4464', 'request_id': 'req-ed7934aa-9977-4705-b04a-1d736873fb95'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:06.071138 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ed7934aa-9977-4705-b04a-1d736873fb95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53a25412091c4285a3124433e04f4464', 'request_id': 'req-ed7934aa-9977-4705-b04a-1d736873fb95'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:06.083097 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ed7934aa-9977-4705-b04a-1d736873fb95 admin admin] The token cache is being invalidate because project 53a25412091c4285a3124433e04f4464 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:08:06.083387 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ed7934aa-9977-4705-b04a-1d736873fb95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:06.084028 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ed7934aa-9977-4705-b04a-1d736873fb95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:06.085625 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1105/2207] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:08:05 2022] DELETE /identity/v3/projects/53a25412091c4285a3124433e04f4464 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:06.089604 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b413b-1c7a-4ae2-970f-eba76b6197a7 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:06.090971 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-156b413b-1c7a-4ae2-970f-eba76b6197a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:06.104211 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b413b-1c7a-4ae2-970f-eba76b6197a7 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:06.161865 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8320f6ae-c5ce-4e48-bb8f-1beac5dcb094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:06.162606 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320f6ae-c5ce-4e48-bb8f-1beac5dcb094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:06.163924 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320f6ae-c5ce-4e48-bb8f-1beac5dcb094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:06.163924 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320f6ae-c5ce-4e48-bb8f-1beac5dcb094 admin admin] PATH_INFO: `/v3/users/fab792fd0f86422cb5053289d173b669` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:06.164713 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8320f6ae-c5ce-4e48-bb8f-1beac5dcb094 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fab792fd0f86422cb5053289d173b669)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:06.166343 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8320f6ae-c5ce-4e48-bb8f-1beac5dcb094 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:06.337225 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8320f6ae-c5ce-4e48-bb8f-1beac5dcb094 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fab792fd0f86422cb5053289d173b669'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:06.337862 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8320f6ae-c5ce-4e48-bb8f-1beac5dcb094 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fab792fd0f86422cb5053289d173b669'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:06.344838 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8320f6ae-c5ce-4e48-bb8f-1beac5dcb094 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fab792fd0f86422cb5053289d173b669'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:06.444929 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8320f6ae-c5ce-4e48-bb8f-1beac5dcb094 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fab792fd0f86422cb5053289d173b669'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:06.445617 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8320f6ae-c5ce-4e48-bb8f-1beac5dcb094 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fab792fd0f86422cb5053289d173b669'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:06.464714 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1103/2208] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:08:06 2022] DELETE /identity/v3/users/fab792fd0f86422cb5053289d173b669 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:06.727516 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:06.923233 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7934aa-9977-4705-b04a-1d736873fb95 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:06.931627 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ed7934aa-9977-4705-b04a-1d736873fb95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:06.943669 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7934aa-9977-4705-b04a-1d736873fb95 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:07.008346 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22ccce3-afda-4cf2-a33d-6f7d20f97a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:07.009709 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ccce3-afda-4cf2-a33d-6f7d20f97a12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:07.009993 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ccce3-afda-4cf2-a33d-6f7d20f97a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:07.010273 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ccce3-afda-4cf2-a33d-6f7d20f97a12 admin admin] PATH_INFO: `/v3/projects/55d7243f696a44f98845e8e9ff1aaa10` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:07.011445 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22ccce3-afda-4cf2-a33d-6f7d20f97a12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55d7243f696a44f98845e8e9ff1aaa10)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:07.013656 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22ccce3-afda-4cf2-a33d-6f7d20f97a12 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:07.078280 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:07.165793 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a22ccce3-afda-4cf2-a33d-6f7d20f97a12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55d7243f696a44f98845e8e9ff1aaa10', 'request_id': 'req-a22ccce3-afda-4cf2-a33d-6f7d20f97a12'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:07.272523 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a22ccce3-afda-4cf2-a33d-6f7d20f97a12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55d7243f696a44f98845e8e9ff1aaa10', 'request_id': 'req-a22ccce3-afda-4cf2-a33d-6f7d20f97a12'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:07.278822 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a22ccce3-afda-4cf2-a33d-6f7d20f97a12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55d7243f696a44f98845e8e9ff1aaa10', 'request_id': 'req-a22ccce3-afda-4cf2-a33d-6f7d20f97a12'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:07.290317 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a22ccce3-afda-4cf2-a33d-6f7d20f97a12 admin admin] The token cache is being invalidate because project 55d7243f696a44f98845e8e9ff1aaa10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:08:07.290615 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a22ccce3-afda-4cf2-a33d-6f7d20f97a12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:07.291329 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a22ccce3-afda-4cf2-a33d-6f7d20f97a12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:07.292842 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1106/2209] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:08:06 2022] DELETE /identity/v3/projects/55d7243f696a44f98845e8e9ff1aaa10 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:07.297772 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbaafb3-e1ab-4918-b0e9-5ba69e234b27 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:07.300711 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbaafb3-e1ab-4918-b0e9-5ba69e234b27 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:07.300711 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbaafb3-e1ab-4918-b0e9-5ba69e234b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:07.321400 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-cfbaafb3-e1ab-4918-b0e9-5ba69e234b27 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:07.322656 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-cfbaafb3-e1ab-4918-b0e9-5ba69e234b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:07.385738 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1104/2210] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:07.393913 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855a02b-aa4e-4579-aa52-e7d766037620 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:07.394197 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855a02b-aa4e-4579-aa52-e7d766037620 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:07.394468 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855a02b-aa4e-4579-aa52-e7d766037620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:07.415209 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-a855a02b-aa4e-4579-aa52-e7d766037620 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:07.416405 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a855a02b-aa4e-4579-aa52-e7d766037620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:07.424205 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1107/2211] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:07.428407 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbaafb3-e1ab-4918-b0e9-5ba69e234b27 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:07.438092 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbaafb3-e1ab-4918-b0e9-5ba69e234b27 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:07.443164 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aed3ca-2771-4570-920e-1f9e30ebbacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:07.443757 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aed3ca-2771-4570-920e-1f9e30ebbacf admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:07.444039 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aed3ca-2771-4570-920e-1f9e30ebbacf admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:07.444312 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aed3ca-2771-4570-920e-1f9e30ebbacf admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:07.445238 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aed3ca-2771-4570-920e-1f9e30ebbacf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:08:07.445543 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aed3ca-2771-4570-920e-1f9e30ebbacf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:07.447135 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aed3ca-2771-4570-920e-1f9e30ebbacf admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:07.459772 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1105/2212] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:08:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 01:08:07.464105 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a855a02b-aa4e-4579-aa52-e7d766037620 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:07.474578 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a855a02b-aa4e-4579-aa52-e7d766037620 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:07.480823 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eeed00-374f-4f36-92cb-c3e32dbb2b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:07.480823 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eeed00-374f-4f36-92cb-c3e32dbb2b22 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:07.480823 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eeed00-374f-4f36-92cb-c3e32dbb2b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:07.480823 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eeed00-374f-4f36-92cb-c3e32dbb2b22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:07.481148 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eeed00-374f-4f36-92cb-c3e32dbb2b22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:07.483129 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eeed00-374f-4f36-92cb-c3e32dbb2b22 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:07.587921 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1108/2213] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:08:07 2022] POST /identity/v3/projects => generated 369 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:08:07.592340 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aed3ca-2771-4570-920e-1f9e30ebbacf admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:07.602602 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aed3ca-2771-4570-920e-1f9e30ebbacf admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:07.659092 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486a2ec0-0aa9-4f7e-b98c-8ccb15f058df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:07.659702 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a2ec0-0aa9-4f7e-b98c-8ccb15f058df admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:07.659985 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a2ec0-0aa9-4f7e-b98c-8ccb15f058df admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:07.660267 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a2ec0-0aa9-4f7e-b98c-8ccb15f058df admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:07.660931 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486a2ec0-0aa9-4f7e-b98c-8ccb15f058df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:07.662848 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486a2ec0-0aa9-4f7e-b98c-8ccb15f058df admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:07.735089 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1106/2214] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:08:07 2022] POST /identity/v3/users => generated 285 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:08:07.736902 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eeed00-374f-4f36-92cb-c3e32dbb2b22 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:07.748296 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eeed00-374f-4f36-92cb-c3e32dbb2b22 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:07.752984 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3302200e-98bb-49d6-b783-d2fdbbefdc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:07.753583 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3302200e-98bb-49d6-b783-d2fdbbefdc99 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:07.753862 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3302200e-98bb-49d6-b783-d2fdbbefdc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:07.754136 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3302200e-98bb-49d6-b783-d2fdbbefdc99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:07.755326 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3302200e-98bb-49d6-b783-d2fdbbefdc99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:07.756949 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3302200e-98bb-49d6-b783-d2fdbbefdc99 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:07.767212 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1109/2215] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:08:07.771370 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486a2ec0-0aa9-4f7e-b98c-8ccb15f058df admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:07.782673 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486a2ec0-0aa9-4f7e-b98c-8ccb15f058df admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:07.788000 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e0fd45-1ff8-4cf9-bf0f-93c2b1b6fe71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:07.788633 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0fd45-1ff8-4cf9-bf0f-93c2b1b6fe71 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:07.788917 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0fd45-1ff8-4cf9-bf0f-93c2b1b6fe71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:07.789195 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0fd45-1ff8-4cf9-bf0f-93c2b1b6fe71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:07.789844 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e0fd45-1ff8-4cf9-bf0f-93c2b1b6fe71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:07.791445 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e0fd45-1ff8-4cf9-bf0f-93c2b1b6fe71 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:07.803632 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1107/2216] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:08:07.807880 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3302200e-98bb-49d6-b783-d2fdbbefdc99 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:07.818553 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3302200e-98bb-49d6-b783-d2fdbbefdc99 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:07.823320 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9bf223-c7e5-4485-90df-0abed6b3ff70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:07.824036 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9bf223-c7e5-4485-90df-0abed6b3ff70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:07.824305 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9bf223-c7e5-4485-90df-0abed6b3ff70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:07.824784 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9bf223-c7e5-4485-90df-0abed6b3ff70 admin admin] PATH_INFO: `/v3/projects/54588287e5e440c9ba341826fa75870f/users/9af5aec0e67f41d6b356af7859708b85/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:07.841790 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9bf223-c7e5-4485-90df-0abed6b3ff70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54588287e5e440c9ba341826fa75870f, user_id=9af5aec0e67f41d6b356af7859708b85, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:07.844089 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9bf223-c7e5-4485-90df-0abed6b3ff70 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:07.919945 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1110/2217] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:08:07 2022] PUT /identity/v3/projects/54588287e5e440c9ba341826fa75870f/users/9af5aec0e67f41d6b356af7859708b85/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:07.932399 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746711e7-f9c9-4d90-8d07-143781b930a4 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:07.932710 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746711e7-f9c9-4d90-8d07-143781b930a4 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:07.932985 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746711e7-f9c9-4d90-8d07-143781b930a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:07.961877 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-746711e7-f9c9-4d90-8d07-143781b930a4 None None] MFA Rules not processed for user `9af5aec0e67f41d6b356af7859708b85`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:07.963075 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-746711e7-f9c9-4d90-8d07-143781b930a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:08.051079 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1108/2218] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:08.074601 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9bf223-c7e5-4485-90df-0abed6b3ff70 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:08.076983 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2d9bf223-c7e5-4485-90df-0abed6b3ff70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:08.199447 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9bf223-c7e5-4485-90df-0abed6b3ff70 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:08.209202 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add5b59-3c71-4074-80b0-ac73537fe032 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1179503bdab547c99175a27b7ce27b9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:08.210119 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add5b59-3c71-4074-80b0-ac73537fe032 service glance] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:08.210414 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add5b59-3c71-4074-80b0-ac73537fe032 service glance] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:08.216529 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add5b59-3c71-4074-80b0-ac73537fe032 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:08.225084 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3add5b59-3c71-4074-80b0-ac73537fe032 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:08.227415 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3add5b59-3c71-4074-80b0-ac73537fe032 service glance] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:08.252925 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1111/2219] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:08:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:08:08.632605 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:08.912926 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:09.146706 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746711e7-f9c9-4d90-8d07-143781b930a4 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:09.148179 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-746711e7-f9c9-4d90-8d07-143781b930a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:09.215820 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746711e7-f9c9-4d90-8d07-143781b930a4 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:09.224904 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30acb528-2358-43a1-9681-9a60af58cbc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:09.224904 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30acb528-2358-43a1-9681-9a60af58cbc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:09.224904 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30acb528-2358-43a1-9681-9a60af58cbc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:09.224904 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30acb528-2358-43a1-9681-9a60af58cbc8 admin admin] PATH_INFO: `/v3/users/bdaf151db2884612af5039fabd124431` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:09.225329 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30acb528-2358-43a1-9681-9a60af58cbc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdaf151db2884612af5039fabd124431)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:09.226443 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30acb528-2358-43a1-9681-9a60af58cbc8 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:09.599644 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-30acb528-2358-43a1-9681-9a60af58cbc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdaf151db2884612af5039fabd124431'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:09.600335 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-30acb528-2358-43a1-9681-9a60af58cbc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdaf151db2884612af5039fabd124431'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:09.608808 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-30acb528-2358-43a1-9681-9a60af58cbc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdaf151db2884612af5039fabd124431'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:09.687476 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-30acb528-2358-43a1-9681-9a60af58cbc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdaf151db2884612af5039fabd124431'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:09.688090 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-30acb528-2358-43a1-9681-9a60af58cbc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdaf151db2884612af5039fabd124431'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:09.709398 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1109/2220] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:08:09 2022] DELETE /identity/v3/users/bdaf151db2884612af5039fabd124431 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:09.713965 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b863bb-67f1-4262-aee7-4951f4a15012 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:09.714386 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b863bb-67f1-4262-aee7-4951f4a15012 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:09.714816 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b863bb-67f1-4262-aee7-4951f4a15012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:09.733927 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-34b863bb-67f1-4262-aee7-4951f4a15012 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:09.735144 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-34b863bb-67f1-4262-aee7-4951f4a15012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:09.767215 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:09.794980 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1112/2221] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:09.805937 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30acb528-2358-43a1-9681-9a60af58cbc8 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:09.807407 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-30acb528-2358-43a1-9681-9a60af58cbc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:09.868791 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30acb528-2358-43a1-9681-9a60af58cbc8 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:09.872509 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798588f-948e-41ae-9ba0-a2c87da56f8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:09.873341 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798588f-948e-41ae-9ba0-a2c87da56f8d service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:09.873564 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798588f-948e-41ae-9ba0-a2c87da56f8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:09.873828 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798588f-948e-41ae-9ba0-a2c87da56f8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:09.883549 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2798588f-948e-41ae-9ba0-a2c87da56f8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:09.885032 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2798588f-948e-41ae-9ba0-a2c87da56f8d service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:09.894565 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1110/2222] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:08:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:08:10.234111 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:10.419151 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:10.548638 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b863bb-67f1-4262-aee7-4951f4a15012 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:10.550361 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-34b863bb-67f1-4262-aee7-4951f4a15012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:10.563653 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b863bb-67f1-4262-aee7-4951f4a15012 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:10.569068 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efbe8d7-3b9d-420f-8a8c-a69a1f5d48c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:10.569787 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efbe8d7-3b9d-420f-8a8c-a69a1f5d48c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:10.570072 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efbe8d7-3b9d-420f-8a8c-a69a1f5d48c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:10.570370 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efbe8d7-3b9d-420f-8a8c-a69a1f5d48c6 admin admin] PATH_INFO: `/v3/projects/f31b99bf5a9a48c0a3901675ae796e15` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:10.571551 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efbe8d7-3b9d-420f-8a8c-a69a1f5d48c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f31b99bf5a9a48c0a3901675ae796e15)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:10.573615 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efbe8d7-3b9d-420f-8a8c-a69a1f5d48c6 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:10.704500 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-6efbe8d7-3b9d-420f-8a8c-a69a1f5d48c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f31b99bf5a9a48c0a3901675ae796e15', 'request_id': 'req-6efbe8d7-3b9d-420f-8a8c-a69a1f5d48c6'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:10.791896 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-6efbe8d7-3b9d-420f-8a8c-a69a1f5d48c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f31b99bf5a9a48c0a3901675ae796e15', 'request_id': 'req-6efbe8d7-3b9d-420f-8a8c-a69a1f5d48c6'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:10.799282 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-6efbe8d7-3b9d-420f-8a8c-a69a1f5d48c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f31b99bf5a9a48c0a3901675ae796e15', 'request_id': 'req-6efbe8d7-3b9d-420f-8a8c-a69a1f5d48c6'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:10.814538 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-6efbe8d7-3b9d-420f-8a8c-a69a1f5d48c6 admin admin] The token cache is being invalidate because project f31b99bf5a9a48c0a3901675ae796e15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:08:10.814926 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-6efbe8d7-3b9d-420f-8a8c-a69a1f5d48c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:10.815651 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-6efbe8d7-3b9d-420f-8a8c-a69a1f5d48c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:10.817860 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1113/2223] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:08:10 2022] DELETE /identity/v3/projects/f31b99bf5a9a48c0a3901675ae796e15 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:08:10.822834 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68af84-e0ca-4a1a-8f56-ccab03e2e217 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:10.823219 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68af84-e0ca-4a1a-8f56-ccab03e2e217 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:10.823562 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68af84-e0ca-4a1a-8f56-ccab03e2e217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:10.843122 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-3c68af84-e0ca-4a1a-8f56-ccab03e2e217 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:10.844414 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3c68af84-e0ca-4a1a-8f56-ccab03e2e217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:10.913256 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1111/2224] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:10.920945 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45bc49-e0e2-4f51-ad92-5c5550b77e51 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:10.923029 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45bc49-e0e2-4f51-ad92-5c5550b77e51 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:10.923029 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45bc49-e0e2-4f51-ad92-5c5550b77e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:10.944421 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-8e45bc49-e0e2-4f51-ad92-5c5550b77e51 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:10.945979 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8e45bc49-e0e2-4f51-ad92-5c5550b77e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:10.956178 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1114/2225] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:10.958461 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68af84-e0ca-4a1a-8f56-ccab03e2e217 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:10.968361 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68af84-e0ca-4a1a-8f56-ccab03e2e217 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:10.974222 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e89619-3f9d-4a9c-91d6-65ce0b14dc5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:10.974834 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e89619-3f9d-4a9c-91d6-65ce0b14dc5c admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:10.975235 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e89619-3f9d-4a9c-91d6-65ce0b14dc5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:10.975517 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e89619-3f9d-4a9c-91d6-65ce0b14dc5c admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:10.976198 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e89619-3f9d-4a9c-91d6-65ce0b14dc5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:08:10.976533 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e89619-3f9d-4a9c-91d6-65ce0b14dc5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:10.978850 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e89619-3f9d-4a9c-91d6-65ce0b14dc5c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:10.992326 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1112/2226] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:08:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:08:10.996244 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45bc49-e0e2-4f51-ad92-5c5550b77e51 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:11.006722 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45bc49-e0e2-4f51-ad92-5c5550b77e51 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:11.011680 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb95cf3-a861-4904-a966-88b5d18ff335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:11.012639 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb95cf3-a861-4904-a966-88b5d18ff335 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:11.012914 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb95cf3-a861-4904-a966-88b5d18ff335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:11.013267 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb95cf3-a861-4904-a966-88b5d18ff335 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:11.013943 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb95cf3-a861-4904-a966-88b5d18ff335 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:11.017334 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb95cf3-a861-4904-a966-88b5d18ff335 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:11.112145 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1115/2227] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:08:10 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:08:11.116038 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e89619-3f9d-4a9c-91d6-65ce0b14dc5c admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:11.127107 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e89619-3f9d-4a9c-91d6-65ce0b14dc5c admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:11.208730 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03c83f1-68a0-4407-bc9d-01f9eefeee5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:11.209329 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c83f1-68a0-4407-bc9d-01f9eefeee5a admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:11.209612 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c83f1-68a0-4407-bc9d-01f9eefeee5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:11.209918 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c83f1-68a0-4407-bc9d-01f9eefeee5a admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:11.210552 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03c83f1-68a0-4407-bc9d-01f9eefeee5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:11.212473 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03c83f1-68a0-4407-bc9d-01f9eefeee5a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:11.308424 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1113/2228] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:08:11 2022] POST /identity/v3/users => generated 287 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:08:11.312228 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb95cf3-a861-4904-a966-88b5d18ff335 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:11.323229 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb95cf3-a861-4904-a966-88b5d18ff335 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:11.327914 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105036c1-920a-41a2-9a0f-1c8cb6aae1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:11.328762 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105036c1-920a-41a2-9a0f-1c8cb6aae1a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:11.329044 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105036c1-920a-41a2-9a0f-1c8cb6aae1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:11.329319 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105036c1-920a-41a2-9a0f-1c8cb6aae1a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:11.330132 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105036c1-920a-41a2-9a0f-1c8cb6aae1a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:11.331912 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105036c1-920a-41a2-9a0f-1c8cb6aae1a3 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:11.348380 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1116/2229] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:08:11.349800 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03c83f1-68a0-4407-bc9d-01f9eefeee5a admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:11.359663 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03c83f1-68a0-4407-bc9d-01f9eefeee5a admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:11.366283 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105036c1-920a-41a2-9a0f-1c8cb6aae1a3 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:11.367788 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8200b593-363a-47a4-8891-8f2353e66337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:11.368653 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200b593-363a-47a4-8891-8f2353e66337 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:11.368783 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-105036c1-920a-41a2-9a0f-1c8cb6aae1a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:11.368898 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200b593-363a-47a4-8891-8f2353e66337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:11.368996 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200b593-363a-47a4-8891-8f2353e66337 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:11.369668 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8200b593-363a-47a4-8891-8f2353e66337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:11.371346 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8200b593-363a-47a4-8891-8f2353e66337 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:11.388409 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105036c1-920a-41a2-9a0f-1c8cb6aae1a3 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:11.389789 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1114/2230] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 01:08:11.393619 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8200b593-363a-47a4-8891-8f2353e66337 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:11.397113 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f30e34-3df7-43e4-a5e5-06002d76d93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:11.401959 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f30e34-3df7-43e4-a5e5-06002d76d93b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:11.401959 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f30e34-3df7-43e4-a5e5-06002d76d93b admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:11.401959 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f30e34-3df7-43e4-a5e5-06002d76d93b admin admin] PATH_INFO: `/v3/users/42063d51ae8f4afea4a0426cd7339903` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:11.401959 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f30e34-3df7-43e4-a5e5-06002d76d93b admin admin] RBAC: Authorizing `identity:delete_user(user_id=42063d51ae8f4afea4a0426cd7339903)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:11.401959 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f30e34-3df7-43e4-a5e5-06002d76d93b admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:11.403579 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8200b593-363a-47a4-8891-8f2353e66337 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:11.409601 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea3aea-81f2-42eb-9214-93b0dd0f35af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:11.410211 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea3aea-81f2-42eb-9214-93b0dd0f35af admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:11.410523 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea3aea-81f2-42eb-9214-93b0dd0f35af admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:11.410848 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea3aea-81f2-42eb-9214-93b0dd0f35af admin admin] PATH_INFO: `/v3/projects/7d5501935ac8454c9f1f89cd8383c7bb/users/9022068567224588b551bba6738e43b1/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:11.426763 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cea3aea-81f2-42eb-9214-93b0dd0f35af admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d5501935ac8454c9f1f89cd8383c7bb, user_id=9022068567224588b551bba6738e43b1, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:11.430693 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cea3aea-81f2-42eb-9214-93b0dd0f35af admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:11.530069 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1115/2231] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:08:11 2022] PUT /identity/v3/projects/7d5501935ac8454c9f1f89cd8383c7bb/users/9022068567224588b551bba6738e43b1/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:11.541039 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd95c8d-208b-4a8d-a323-1bd806678e6b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:11.541326 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd95c8d-208b-4a8d-a323-1bd806678e6b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:11.541600 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd95c8d-208b-4a8d-a323-1bd806678e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:11.568006 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-3bd95c8d-208b-4a8d-a323-1bd806678e6b None None] MFA Rules not processed for user `9022068567224588b551bba6738e43b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:11.569388 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3bd95c8d-208b-4a8d-a323-1bd806678e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:11.638035 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1116/2232] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:08:11.649014 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd95c8d-208b-4a8d-a323-1bd806678e6b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:11.650360 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3bd95c8d-208b-4a8d-a323-1bd806678e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:11.670121 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a7f30e34-3df7-43e4-a5e5-06002d76d93b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42063d51ae8f4afea4a0426cd7339903'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:11.670953 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a7f30e34-3df7-43e4-a5e5-06002d76d93b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42063d51ae8f4afea4a0426cd7339903'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:11.728165 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd95c8d-208b-4a8d-a323-1bd806678e6b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:11.734764 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf0201-efcb-4192-97ff-ee93826ef1f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:11.735424 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf0201-efcb-4192-97ff-ee93826ef1f0 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:11.735745 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf0201-efcb-4192-97ff-ee93826ef1f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:11.736029 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf0201-efcb-4192-97ff-ee93826ef1f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:11.737938 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-aadf0201-efcb-4192-97ff-ee93826ef1f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:11.741258 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a7f30e34-3df7-43e4-a5e5-06002d76d93b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42063d51ae8f4afea4a0426cd7339903'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:11.742157 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a7f30e34-3df7-43e4-a5e5-06002d76d93b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42063d51ae8f4afea4a0426cd7339903'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:11.751775 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf0201-efcb-4192-97ff-ee93826ef1f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:11.753403 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf0201-efcb-4192-97ff-ee93826ef1f0 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:11.754953 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a7f30e34-3df7-43e4-a5e5-06002d76d93b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42063d51ae8f4afea4a0426cd7339903'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:11.763005 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1117/2233] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:08:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 01:08:11.767942 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1117/2234] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:08:11 2022] DELETE /identity/v3/users/42063d51ae8f4afea4a0426cd7339903 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:11.773988 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d9093-0447-48be-9b11-11999aa424b8 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:11.775031 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d9093-0447-48be-9b11-11999aa424b8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:11.775328 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d9093-0447-48be-9b11-11999aa424b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:11.802927 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-593d9093-0447-48be-9b11-11999aa424b8 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:11.802927 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-593d9093-0447-48be-9b11-11999aa424b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:11.838496 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:11.870931 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1118/2235] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:11.878588 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f30e34-3df7-43e4-a5e5-06002d76d93b admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:11.880170 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a7f30e34-3df7-43e4-a5e5-06002d76d93b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:11.900108 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:11.993201 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f30e34-3df7-43e4-a5e5-06002d76d93b admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:11.998684 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde8b6d-0f7b-4142-b6d5-fd26ed81c54e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:11.999318 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde8b6d-0f7b-4142-b6d5-fd26ed81c54e service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:11.999603 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde8b6d-0f7b-4142-b6d5-fd26ed81c54e service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:11.999880 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde8b6d-0f7b-4142-b6d5-fd26ed81c54e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:12.016878 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cde8b6d-0f7b-4142-b6d5-fd26ed81c54e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:12.020752 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cde8b6d-0f7b-4142-b6d5-fd26ed81c54e service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:12.035073 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1118/2236] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:08:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:08:12.053576 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d9093-0447-48be-9b11-11999aa424b8 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:12.053576 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-593d9093-0447-48be-9b11-11999aa424b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:12.066710 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d9093-0447-48be-9b11-11999aa424b8 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:12.072910 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55f0ca-f486-4694-b9ed-26a81c8f0c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:12.073528 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55f0ca-f486-4694-b9ed-26a81c8f0c1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:12.073879 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55f0ca-f486-4694-b9ed-26a81c8f0c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:12.074223 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55f0ca-f486-4694-b9ed-26a81c8f0c1b admin admin] PATH_INFO: `/v3/users/9022068567224588b551bba6738e43b1` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:12.075460 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55f0ca-f486-4694-b9ed-26a81c8f0c1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9022068567224588b551bba6738e43b1)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:12.077830 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55f0ca-f486-4694-b9ed-26a81c8f0c1b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:12.300829 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6c55f0ca-f486-4694-b9ed-26a81c8f0c1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9022068567224588b551bba6738e43b1'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:12.301642 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6c55f0ca-f486-4694-b9ed-26a81c8f0c1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9022068567224588b551bba6738e43b1'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:12.308991 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6c55f0ca-f486-4694-b9ed-26a81c8f0c1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9022068567224588b551bba6738e43b1'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:12.394293 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6c55f0ca-f486-4694-b9ed-26a81c8f0c1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9022068567224588b551bba6738e43b1'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:12.394293 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6c55f0ca-f486-4694-b9ed-26a81c8f0c1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9022068567224588b551bba6738e43b1'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:12.411624 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1119/2237] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:08:12 2022] DELETE /identity/v3/users/9022068567224588b551bba6738e43b1 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:12.416764 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163c237-7c82-4dfc-b4f7-094faf2f6469 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:12.416764 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163c237-7c82-4dfc-b4f7-094faf2f6469 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:12.417009 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163c237-7c82-4dfc-b4f7-094faf2f6469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:12.436567 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-4163c237-7c82-4dfc-b4f7-094faf2f6469 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:12.439727 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-4163c237-7c82-4dfc-b4f7-094faf2f6469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:12.500593 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1119/2238] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:12.511378 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55f0ca-f486-4694-b9ed-26a81c8f0c1b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:12.513167 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6c55f0ca-f486-4694-b9ed-26a81c8f0c1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:12.563896 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163c237-7c82-4dfc-b4f7-094faf2f6469 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:12.565644 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-4163c237-7c82-4dfc-b4f7-094faf2f6469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:12.577094 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55f0ca-f486-4694-b9ed-26a81c8f0c1b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:12.579796 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163c237-7c82-4dfc-b4f7-094faf2f6469 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:12.584014 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee05450-a973-443e-88a8-47c1029a340d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:12.584014 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee05450-a973-443e-88a8-47c1029a340d service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:12.584014 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee05450-a973-443e-88a8-47c1029a340d service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:12.584014 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee05450-a973-443e-88a8-47c1029a340d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:12.585737 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d832c7-7f07-49a4-82ab-131e89809ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:12.586364 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d832c7-7f07-49a4-82ab-131e89809ca1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:12.586671 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d832c7-7f07-49a4-82ab-131e89809ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:12.586945 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d832c7-7f07-49a4-82ab-131e89809ca1 admin admin] PATH_INFO: `/v3/projects/ba08a6957ed346d0b747835e3dc15696` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:12.588173 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d832c7-7f07-49a4-82ab-131e89809ca1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba08a6957ed346d0b747835e3dc15696)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:12.590453 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d832c7-7f07-49a4-82ab-131e89809ca1 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:12.600848 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee05450-a973-443e-88a8-47c1029a340d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:12.602802 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee05450-a973-443e-88a8-47c1029a340d service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:12.613369 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1120/2239] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:08:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:08:12.832686 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:12.852133 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-c1d832c7-7f07-49a4-82ab-131e89809ca1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba08a6957ed346d0b747835e3dc15696', 'request_id': 'req-c1d832c7-7f07-49a4-82ab-131e89809ca1'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:12.938519 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-c1d832c7-7f07-49a4-82ab-131e89809ca1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba08a6957ed346d0b747835e3dc15696', 'request_id': 'req-c1d832c7-7f07-49a4-82ab-131e89809ca1'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:12.947776 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-c1d832c7-7f07-49a4-82ab-131e89809ca1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba08a6957ed346d0b747835e3dc15696', 'request_id': 'req-c1d832c7-7f07-49a4-82ab-131e89809ca1'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:12.963024 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-c1d832c7-7f07-49a4-82ab-131e89809ca1 admin admin] The token cache is being invalidate because project ba08a6957ed346d0b747835e3dc15696 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:08:12.963318 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-c1d832c7-7f07-49a4-82ab-131e89809ca1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:12.964177 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-c1d832c7-7f07-49a4-82ab-131e89809ca1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:12.965730 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1120/2240] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:08:12 2022] DELETE /identity/v3/projects/ba08a6957ed346d0b747835e3dc15696 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:12.972486 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee05450-a973-443e-88a8-47c1029a340d service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:12.980662 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee05450-a973-443e-88a8-47c1029a340d service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:13.048413 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ad259-2fa2-44d9-a51f-5c96ec6b959d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:13.049080 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ad259-2fa2-44d9-a51f-5c96ec6b959d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:13.049374 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ad259-2fa2-44d9-a51f-5c96ec6b959d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:13.049621 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ad259-2fa2-44d9-a51f-5c96ec6b959d admin admin] PATH_INFO: `/v3/users/05979affd0924ba7a7583171e05efc60` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:13.050722 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ad259-2fa2-44d9-a51f-5c96ec6b959d admin admin] RBAC: Authorizing `identity:delete_user(user_id=05979affd0924ba7a7583171e05efc60)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:13.053258 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ad259-2fa2-44d9-a51f-5c96ec6b959d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:13.120819 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d832c7-7f07-49a4-82ab-131e89809ca1 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:13.122308 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c1d832c7-7f07-49a4-82ab-131e89809ca1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:13.146756 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d832c7-7f07-49a4-82ab-131e89809ca1 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:13.153306 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b8d2c-c189-455a-810c-d893320a62ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:13.153306 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b8d2c-c189-455a-810c-d893320a62ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:13.153306 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b8d2c-c189-455a-810c-d893320a62ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:13.153306 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b8d2c-c189-455a-810c-d893320a62ad admin admin] PATH_INFO: `/v3/projects/7d5501935ac8454c9f1f89cd8383c7bb` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:13.154016 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596b8d2c-c189-455a-810c-d893320a62ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d5501935ac8454c9f1f89cd8383c7bb)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:13.155911 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596b8d2c-c189-455a-810c-d893320a62ad admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:13.317180 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:13.333736 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a55ad259-2fa2-44d9-a51f-5c96ec6b959d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05979affd0924ba7a7583171e05efc60'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:13.334717 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a55ad259-2fa2-44d9-a51f-5c96ec6b959d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05979affd0924ba7a7583171e05efc60'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:13.335498 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-596b8d2c-c189-455a-810c-d893320a62ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d5501935ac8454c9f1f89cd8383c7bb', 'request_id': 'req-596b8d2c-c189-455a-810c-d893320a62ad'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:13.343112 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a55ad259-2fa2-44d9-a51f-5c96ec6b959d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05979affd0924ba7a7583171e05efc60'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:13.483737 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-596b8d2c-c189-455a-810c-d893320a62ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d5501935ac8454c9f1f89cd8383c7bb', 'request_id': 'req-596b8d2c-c189-455a-810c-d893320a62ad'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:13.493377 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-596b8d2c-c189-455a-810c-d893320a62ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d5501935ac8454c9f1f89cd8383c7bb', 'request_id': 'req-596b8d2c-c189-455a-810c-d893320a62ad'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:13.502643 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-596b8d2c-c189-455a-810c-d893320a62ad admin admin] The token cache is being invalidate because project 7d5501935ac8454c9f1f89cd8383c7bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:08:13.502928 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-596b8d2c-c189-455a-810c-d893320a62ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:13.503598 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-596b8d2c-c189-455a-810c-d893320a62ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:13.505326 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1121/2241] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:08:13 2022] DELETE /identity/v3/projects/7d5501935ac8454c9f1f89cd8383c7bb => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:08:13.509959 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a76877-12ac-403d-b41b-e5b926e1ab52 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:13.510246 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a76877-12ac-403d-b41b-e5b926e1ab52 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:13.510539 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a76877-12ac-403d-b41b-e5b926e1ab52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:13.531668 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-a2a76877-12ac-403d-b41b-e5b926e1ab52 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:13.532909 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a2a76877-12ac-403d-b41b-e5b926e1ab52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:13.592032 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1122/2242] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:13.600363 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a55ad259-2fa2-44d9-a51f-5c96ec6b959d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05979affd0924ba7a7583171e05efc60'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:13.601443 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366403bb-93fa-4f09-87b7-338219474201 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:13.601550 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a55ad259-2fa2-44d9-a51f-5c96ec6b959d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05979affd0924ba7a7583171e05efc60'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:13.601734 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366403bb-93fa-4f09-87b7-338219474201 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:13.602007 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366403bb-93fa-4f09-87b7-338219474201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:13.619863 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1121/2243] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:08:12 2022] DELETE /identity/v3/users/05979affd0924ba7a7583171e05efc60 => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:13.621921 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-366403bb-93fa-4f09-87b7-338219474201 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:13.623170 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-366403bb-93fa-4f09-87b7-338219474201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:13.687409 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1123/2244] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:08:13.691902 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ad259-2fa2-44d9-a51f-5c96ec6b959d admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:13.693375 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a55ad259-2fa2-44d9-a51f-5c96ec6b959d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:13.706088 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ad259-2fa2-44d9-a51f-5c96ec6b959d admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:13.711515 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3d22b-ff0b-4736-9405-6f65b9c13354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:13.712120 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3d22b-ff0b-4736-9405-6f65b9c13354 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:13.712398 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3d22b-ff0b-4736-9405-6f65b9c13354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:13.712703 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3d22b-ff0b-4736-9405-6f65b9c13354 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:13.713348 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e3d22b-ff0b-4736-9405-6f65b9c13354 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:08:13.713675 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e3d22b-ff0b-4736-9405-6f65b9c13354 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:13.715300 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e3d22b-ff0b-4736-9405-6f65b9c13354 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:13.729581 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1122/2245] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:08:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:08:13.733071 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366403bb-93fa-4f09-87b7-338219474201 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:13.742463 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366403bb-93fa-4f09-87b7-338219474201 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:13.747179 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d86ca-e01e-4320-a944-ff6bf38a205c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:13.747782 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d86ca-e01e-4320-a944-ff6bf38a205c admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:13.748063 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d86ca-e01e-4320-a944-ff6bf38a205c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:13.748340 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d86ca-e01e-4320-a944-ff6bf38a205c admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:13.749047 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d86ca-e01e-4320-a944-ff6bf38a205c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:13.750986 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d86ca-e01e-4320-a944-ff6bf38a205c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:13.884060 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1124/2246] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:08:13 2022] POST /identity/v3/projects => generated 363 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:08:13.888270 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3d22b-ff0b-4736-9405-6f65b9c13354 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:13.899015 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3d22b-ff0b-4736-9405-6f65b9c13354 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:13.958619 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d52ad2-9a0b-4330-86b2-a5cccacb1892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:13.959224 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d52ad2-9a0b-4330-86b2-a5cccacb1892 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:13.959505 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d52ad2-9a0b-4330-86b2-a5cccacb1892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:13.959909 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d52ad2-9a0b-4330-86b2-a5cccacb1892 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:13.960480 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d52ad2-9a0b-4330-86b2-a5cccacb1892 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:13.962370 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d52ad2-9a0b-4330-86b2-a5cccacb1892 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:14.103613 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1123/2247] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:08:13 2022] POST /identity/v3/users => generated 282 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:08:14.104799 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d86ca-e01e-4320-a944-ff6bf38a205c admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:14.106194 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d52ad2-9a0b-4330-86b2-a5cccacb1892 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:14.107976 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d00d86ca-e01e-4320-a944-ff6bf38a205c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:14.119486 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d52ad2-9a0b-4330-86b2-a5cccacb1892 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:14.128849 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d86ca-e01e-4320-a944-ff6bf38a205c admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:14.131887 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4bde88-5ff7-4746-91c7-dbc71c6d1ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:14.132524 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bde88-5ff7-4746-91c7-dbc71c6d1ac8 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:14.133097 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bde88-5ff7-4746-91c7-dbc71c6d1ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:14.133097 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bde88-5ff7-4746-91c7-dbc71c6d1ac8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:14.133725 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4bde88-5ff7-4746-91c7-dbc71c6d1ac8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:14.136055 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4bde88-5ff7-4746-91c7-dbc71c6d1ac8 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:14.144717 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d22995-6a55-400d-933d-0ab024c01715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:14.146231 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d22995-6a55-400d-933d-0ab024c01715 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:14.146517 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d22995-6a55-400d-933d-0ab024c01715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:14.146802 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d22995-6a55-400d-933d-0ab024c01715 admin admin] PATH_INFO: `/v3/projects/5d7be4e7725d4f1fa9c10fc1d26a70b2` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:14.148219 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d22995-6a55-400d-933d-0ab024c01715 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d7be4e7725d4f1fa9c10fc1d26a70b2)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:14.150886 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d22995-6a55-400d-933d-0ab024c01715 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:14.159343 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1124/2248] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:08:14.161217 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4bde88-5ff7-4746-91c7-dbc71c6d1ac8 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:14.172298 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4bde88-5ff7-4746-91c7-dbc71c6d1ac8 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:14.177765 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d99420-94e5-404c-9abc-54cb6a1a8f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:14.178435 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d99420-94e5-404c-9abc-54cb6a1a8f01 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:14.178706 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d99420-94e5-404c-9abc-54cb6a1a8f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:14.179006 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d99420-94e5-404c-9abc-54cb6a1a8f01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:14.179652 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d99420-94e5-404c-9abc-54cb6a1a8f01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:14.181302 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d99420-94e5-404c-9abc-54cb6a1a8f01 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:14.194435 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1125/2249] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:08:14.200932 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d99420-94e5-404c-9abc-54cb6a1a8f01 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:14.213039 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d99420-94e5-404c-9abc-54cb6a1a8f01 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:14.217874 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238d4512-3ec6-4eea-a4e6-36c644f7afcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:14.218517 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d4512-3ec6-4eea-a4e6-36c644f7afcb admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:14.218833 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d4512-3ec6-4eea-a4e6-36c644f7afcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:14.226150 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d4512-3ec6-4eea-a4e6-36c644f7afcb admin admin] PATH_INFO: `/v3/projects/a2242b4c450d4a0cb161a78483fd9c93/users/349292c48f0c4f4195e8970f9137de77/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:14.236723 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238d4512-3ec6-4eea-a4e6-36c644f7afcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2242b4c450d4a0cb161a78483fd9c93, user_id=349292c48f0c4f4195e8970f9137de77, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:14.238609 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238d4512-3ec6-4eea-a4e6-36c644f7afcb admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:14.389246 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1126/2250] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:08:14 2022] PUT /identity/v3/projects/a2242b4c450d4a0cb161a78483fd9c93/users/349292c48f0c4f4195e8970f9137de77/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:08:14.397944 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-98d22995-6a55-400d-933d-0ab024c01715 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d7be4e7725d4f1fa9c10fc1d26a70b2', 'request_id': 'req-98d22995-6a55-400d-933d-0ab024c01715'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:14.401868 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c98e6-66a6-47d7-865f-7f0464aa91e1 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:14.402821 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c98e6-66a6-47d7-865f-7f0464aa91e1 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:14.402821 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c98e6-66a6-47d7-865f-7f0464aa91e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:14.435863 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-176c98e6-66a6-47d7-865f-7f0464aa91e1 None None] MFA Rules not processed for user `349292c48f0c4f4195e8970f9137de77`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:14.437139 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-176c98e6-66a6-47d7-865f-7f0464aa91e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:14.505893 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1127/2251] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:08:14.510933 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c98e6-66a6-47d7-865f-7f0464aa91e1 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:14.517875 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-98d22995-6a55-400d-933d-0ab024c01715 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d7be4e7725d4f1fa9c10fc1d26a70b2', 'request_id': 'req-98d22995-6a55-400d-933d-0ab024c01715'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:14.530918 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-98d22995-6a55-400d-933d-0ab024c01715 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d7be4e7725d4f1fa9c10fc1d26a70b2', 'request_id': 'req-98d22995-6a55-400d-933d-0ab024c01715'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:14.545305 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-98d22995-6a55-400d-933d-0ab024c01715 admin admin] The token cache is being invalidate because project 5d7be4e7725d4f1fa9c10fc1d26a70b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:08:14.549758 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-98d22995-6a55-400d-933d-0ab024c01715 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:14.549758 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-98d22995-6a55-400d-933d-0ab024c01715 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:14.549758 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1125/2252] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:08:14 2022] DELETE /identity/v3/projects/5d7be4e7725d4f1fa9c10fc1d26a70b2 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:14.554004 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa182ccd-87ce-483a-9e88-63f1f517567b None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:14.554519 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa182ccd-87ce-483a-9e88-63f1f517567b None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:14.554899 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa182ccd-87ce-483a-9e88-63f1f517567b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:14.585026 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c98e6-66a6-47d7-865f-7f0464aa91e1 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:14.590082 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b43e0-a170-476a-b0aa-c1263c8a267b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:14.590821 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b43e0-a170-476a-b0aa-c1263c8a267b admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:14.591101 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b43e0-a170-476a-b0aa-c1263c8a267b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:14.591206 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-aa182ccd-87ce-483a-9e88-63f1f517567b None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:14.591387 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b43e0-a170-476a-b0aa-c1263c8a267b admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:14.592138 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48b43e0-a170-476a-b0aa-c1263c8a267b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:14.594200 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48b43e0-a170-476a-b0aa-c1263c8a267b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:14.594387 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-aa182ccd-87ce-483a-9e88-63f1f517567b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:14.607948 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1126/2253] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:08:14.617883 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dba941-c823-469e-afd0-f7c8efa4ecf3 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:14.618393 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dba941-c823-469e-afd0-f7c8efa4ecf3 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:14.618673 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dba941-c823-469e-afd0-f7c8efa4ecf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:14.639718 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-a9dba941-c823-469e-afd0-f7c8efa4ecf3 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:14.641026 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a9dba941-c823-469e-afd0-f7c8efa4ecf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:14.654564 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1127/2254] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:08:14.658985 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dba941-c823-469e-afd0-f7c8efa4ecf3 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:14.668916 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dba941-c823-469e-afd0-f7c8efa4ecf3 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:14.673637 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce68a9-30b0-48b7-8794-8a8b7b134d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:14.674235 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce68a9-30b0-48b7-8794-8a8b7b134d93 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:14.674515 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce68a9-30b0-48b7-8794-8a8b7b134d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:14.674786 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce68a9-30b0-48b7-8794-8a8b7b134d93 admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:14.675505 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce68a9-30b0-48b7-8794-8a8b7b134d93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:08:14.677207 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce68a9-30b0-48b7-8794-8a8b7b134d93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:14.678200 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce68a9-30b0-48b7-8794-8a8b7b134d93 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:14.694632 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1128/2255] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:08:14 2022] POST /identity/v3/projects => generated 365 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:08:14.698928 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1128/2256] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 01:08:14.699071 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b43e0-a170-476a-b0aa-c1263c8a267b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:14.699820 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce68a9-30b0-48b7-8794-8a8b7b134d93 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:14.700120 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a48b43e0-a170-476a-b0aa-c1263c8a267b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:14.710682 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce68a9-30b0-48b7-8794-8a8b7b134d93 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:14.772383 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b43e0-a170-476a-b0aa-c1263c8a267b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:14.782283 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e4e28-4261-4f72-9f18-69c8a616b847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:14.783214 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e4e28-4261-4f72-9f18-69c8a616b847 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:14.783724 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e4e28-4261-4f72-9f18-69c8a616b847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:14.784150 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e4e28-4261-4f72-9f18-69c8a616b847 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:14.784263 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb56415-f2ab-495a-921b-d6e6b2d144b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:14.784970 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb56415-f2ab-495a-921b-d6e6b2d144b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:14.785791 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb56415-f2ab-495a-921b-d6e6b2d144b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:14.786083 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e4e28-4261-4f72-9f18-69c8a616b847 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:14.786179 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb56415-f2ab-495a-921b-d6e6b2d144b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:14.787106 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb56415-f2ab-495a-921b-d6e6b2d144b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:14.788005 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e4e28-4261-4f72-9f18-69c8a616b847 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:14.789219 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb56415-f2ab-495a-921b-d6e6b2d144b1 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:14.913678 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1129/2257] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:08:14 2022] POST /identity/v3/users => generated 289 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:08:14.917378 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e4e28-4261-4f72-9f18-69c8a616b847 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:14.928409 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e4e28-4261-4f72-9f18-69c8a616b847 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:14.933060 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e4057-354d-4001-9dd5-f91992309439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:14.933652 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e4057-354d-4001-9dd5-f91992309439 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:14.933932 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e4057-354d-4001-9dd5-f91992309439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:14.934207 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e4057-354d-4001-9dd5-f91992309439 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:14.934895 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1e4057-354d-4001-9dd5-f91992309439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:14.936495 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1e4057-354d-4001-9dd5-f91992309439 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:14.940286 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:14.945776 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1129/2258] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:08:14 2022] POST /identity/v3/projects => generated 373 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:08:14.956141 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1130/2259] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 01:08:14.956471 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb56415-f2ab-495a-921b-d6e6b2d144b1 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:14.957383 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e4057-354d-4001-9dd5-f91992309439 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:14.969613 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e4057-354d-4001-9dd5-f91992309439 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:15.031873 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:15.034959 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb56415-f2ab-495a-921b-d6e6b2d144b1 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:15.040691 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3f812-4bb6-4eff-9e6d-12c733db459f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:15.041371 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3f812-4bb6-4eff-9e6d-12c733db459f admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:15.041662 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3f812-4bb6-4eff-9e6d-12c733db459f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:15.041767 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512d20f-03db-4029-bd38-3ac8fed7d93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:15.041975 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3f812-4bb6-4eff-9e6d-12c733db459f admin admin] PATH_INFO: `/v3/projects/dee2f21c9543451eae265749755c518a/users/3e2c7194598e4ab7ba4e7ed648f8471b/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:15.042419 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512d20f-03db-4029-bd38-3ac8fed7d93b admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:15.042744 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512d20f-03db-4029-bd38-3ac8fed7d93b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:15.042962 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512d20f-03db-4029-bd38-3ac8fed7d93b admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:15.043587 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b512d20f-03db-4029-bd38-3ac8fed7d93b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:15.046025 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b512d20f-03db-4029-bd38-3ac8fed7d93b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:15.059139 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b3f812-4bb6-4eff-9e6d-12c733db459f admin admin] RBAC: Authorizing `identity:create_grant(project_id=dee2f21c9543451eae265749755c518a, user_id=3e2c7194598e4ab7ba4e7ed648f8471b, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:15.062033 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b3f812-4bb6-4eff-9e6d-12c733db459f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:15.147790 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1131/2260] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:08:14 2022] POST /identity/v3/users => generated 287 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:08:15.152417 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512d20f-03db-4029-bd38-3ac8fed7d93b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:15.168975 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512d20f-03db-4029-bd38-3ac8fed7d93b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:15.172709 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1130/2261] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:08:14 2022] PUT /identity/v3/projects/dee2f21c9543451eae265749755c518a/users/3e2c7194598e4ab7ba4e7ed648f8471b/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:08:15.173886 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3f812-4bb6-4eff-9e6d-12c733db459f admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:15.235187 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe8e98c-0db7-4d77-820e-5a73b9b3b8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:15.236095 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8e98c-0db7-4d77-820e-5a73b9b3b8e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:15.236602 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8e98c-0db7-4d77-820e-5a73b9b3b8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:15.239523 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8e98c-0db7-4d77-820e-5a73b9b3b8e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:15.239523 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe8e98c-0db7-4d77-820e-5a73b9b3b8e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:15.244399 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe8e98c-0db7-4d77-820e-5a73b9b3b8e8 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:15.255844 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1132/2262] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:08:15.261433 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe8e98c-0db7-4d77-820e-5a73b9b3b8e8 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:15.265525 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3f812-4bb6-4eff-9e6d-12c733db459f admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:15.270955 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad211e-e179-4fb0-a613-26a434885bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:15.271694 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad211e-e179-4fb0-a613-26a434885bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:15.272018 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad211e-e179-4fb0-a613-26a434885bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:15.272296 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad211e-e179-4fb0-a613-26a434885bd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:15.272394 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe8e98c-0db7-4d77-820e-5a73b9b3b8e8 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:15.272980 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad211e-e179-4fb0-a613-26a434885bd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:15.274935 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad211e-e179-4fb0-a613-26a434885bd8 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:15.282274 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d1936-1790-449b-97b6-505ab4f9fd18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:15.283064 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d1936-1790-449b-97b6-505ab4f9fd18 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:15.283353 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d1936-1790-449b-97b6-505ab4f9fd18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:15.283627 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d1936-1790-449b-97b6-505ab4f9fd18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:15.284614 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7d1936-1790-449b-97b6-505ab4f9fd18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:15.286886 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7d1936-1790-449b-97b6-505ab4f9fd18 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:15.287684 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1131/2263] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:08:15.291553 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad211e-e179-4fb0-a613-26a434885bd8 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:15.299041 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1133/2264] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:08:15.303647 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad211e-e179-4fb0-a613-26a434885bd8 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:15.304054 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d1936-1790-449b-97b6-505ab4f9fd18 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:15.309312 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b99c89-2fe2-4e6f-8c56-abb5da09915e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:15.309919 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b99c89-2fe2-4e6f-8c56-abb5da09915e admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:15.310206 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b99c89-2fe2-4e6f-8c56-abb5da09915e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:15.310482 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b99c89-2fe2-4e6f-8c56-abb5da09915e admin admin] PATH_INFO: `/v3/projects/dee2f21c9543451eae265749755c518a/users/3e2c7194598e4ab7ba4e7ed648f8471b/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:15.312450 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b99c89-2fe2-4e6f-8c56-abb5da09915e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dee2f21c9543451eae265749755c518a, user_id=3e2c7194598e4ab7ba4e7ed648f8471b, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:15.315153 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b99c89-2fe2-4e6f-8c56-abb5da09915e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:15.316722 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d1936-1790-449b-97b6-505ab4f9fd18 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:15.323544 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3d978-1c4b-454a-ae5c-dfdf9537dbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:15.324452 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3d978-1c4b-454a-ae5c-dfdf9537dbb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:15.324928 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3d978-1c4b-454a-ae5c-dfdf9537dbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:15.325363 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3d978-1c4b-454a-ae5c-dfdf9537dbb8 admin admin] PATH_INFO: `/v3/projects/b14ba9e7b3524b509ef833c2c9bd8067/users/e1b78df6534141d29020d384ddf311b6/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:15.328138 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1132/2265] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:08:15 2022] PUT /identity/v3/projects/dee2f21c9543451eae265749755c518a/users/3e2c7194598e4ab7ba4e7ed648f8471b/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:08:15.339491 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8395b3-0f8e-4460-8702-8bced410ea6a None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:15.339799 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8395b3-0f8e-4460-8702-8bced410ea6a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:15.340290 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8395b3-0f8e-4460-8702-8bced410ea6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:15.343594 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a3d978-1c4b-454a-ae5c-dfdf9537dbb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b14ba9e7b3524b509ef833c2c9bd8067, user_id=e1b78df6534141d29020d384ddf311b6, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:15.346346 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a3d978-1c4b-454a-ae5c-dfdf9537dbb8 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:15.373617 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-0f8395b3-0f8e-4460-8702-8bced410ea6a None None] MFA Rules not processed for user `3e2c7194598e4ab7ba4e7ed648f8471b`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:15.374870 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-0f8395b3-0f8e-4460-8702-8bced410ea6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:15.446377 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1133/2266] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:15 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:08:15.458196 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8395b3-0f8e-4460-8702-8bced410ea6a None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:15.459767 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-0f8395b3-0f8e-4460-8702-8bced410ea6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:15.517863 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8395b3-0f8e-4460-8702-8bced410ea6a None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:15.521701 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5492025-930c-4f59-b40e-816e9c2d3812 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:15.522372 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5492025-930c-4f59-b40e-816e9c2d3812 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:15.522651 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5492025-930c-4f59-b40e-816e9c2d3812 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:15.522968 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5492025-930c-4f59-b40e-816e9c2d3812 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:15.532453 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5492025-930c-4f59-b40e-816e9c2d3812 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:15.533976 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5492025-930c-4f59-b40e-816e9c2d3812 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:15.543814 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1134/2267] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:08:15 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 01:08:15.573350 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1134/2268] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:08:15 2022] PUT /identity/v3/projects/b14ba9e7b3524b509ef833c2c9bd8067/users/e1b78df6534141d29020d384ddf311b6/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:08:15.584422 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ddac0-55cf-41a6-b974-e61740bcb497 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:15.584422 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ddac0-55cf-41a6-b974-e61740bcb497 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:15.584422 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ddac0-55cf-41a6-b974-e61740bcb497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:15.618356 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-d91ddac0-55cf-41a6-b974-e61740bcb497 None None] MFA Rules not processed for user `e1b78df6534141d29020d384ddf311b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:15.619671 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d91ddac0-55cf-41a6-b974-e61740bcb497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:15.687804 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:15.694864 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1135/2269] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:15.739989 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3d978-1c4b-454a-ae5c-dfdf9537dbb8 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:15.799983 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3d978-1c4b-454a-ae5c-dfdf9537dbb8 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:15.804109 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd65c3-19d1-443d-99b3-aac6b7c2ef8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:15.804731 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd65c3-19d1-443d-99b3-aac6b7c2ef8d service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:15.805015 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd65c3-19d1-443d-99b3-aac6b7c2ef8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:15.808012 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd65c3-19d1-443d-99b3-aac6b7c2ef8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:15.812926 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:15.814024 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd65c3-19d1-443d-99b3-aac6b7c2ef8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:15.815448 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd65c3-19d1-443d-99b3-aac6b7c2ef8d service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:15.826959 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1135/2270] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:08:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:08:16.199566 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ddac0-55cf-41a6-b974-e61740bcb497 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:16.200998 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d91ddac0-55cf-41a6-b974-e61740bcb497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:16.213716 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ddac0-55cf-41a6-b974-e61740bcb497 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:16.217374 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e58e28-b4ef-43fc-aa66-fb003b9016aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:16.217978 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e58e28-b4ef-43fc-aa66-fb003b9016aa service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:16.218257 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e58e28-b4ef-43fc-aa66-fb003b9016aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:16.218555 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e58e28-b4ef-43fc-aa66-fb003b9016aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:16.220369 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-11e58e28-b4ef-43fc-aa66-fb003b9016aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:16.287789 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:16.288387 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e58e28-b4ef-43fc-aa66-fb003b9016aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:16.290130 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e58e28-b4ef-43fc-aa66-fb003b9016aa service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:16.300146 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1136/2271] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:08:16 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:08:16.455470 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:16.850013 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:17.102941 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd65c3-19d1-443d-99b3-aac6b7c2ef8d service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:17.182580 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd65c3-19d1-443d-99b3-aac6b7c2ef8d service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:17.188599 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e3e76-d237-4b63-b2f2-9727a2029078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:17.189339 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e3e76-d237-4b63-b2f2-9727a2029078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:17.189631 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e3e76-d237-4b63-b2f2-9727a2029078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:17.190155 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e3e76-d237-4b63-b2f2-9727a2029078 admin admin] PATH_INFO: `/v3/users/349292c48f0c4f4195e8970f9137de77` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:17.192131 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96e3e76-d237-4b63-b2f2-9727a2029078 admin admin] RBAC: Authorizing `identity:delete_user(user_id=349292c48f0c4f4195e8970f9137de77)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:17.196978 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96e3e76-d237-4b63-b2f2-9727a2029078 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:17.247302 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:17.401844 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c96e3e76-d237-4b63-b2f2-9727a2029078 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '349292c48f0c4f4195e8970f9137de77'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:17.402494 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c96e3e76-d237-4b63-b2f2-9727a2029078 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '349292c48f0c4f4195e8970f9137de77'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:17.409474 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c96e3e76-d237-4b63-b2f2-9727a2029078 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '349292c48f0c4f4195e8970f9137de77'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:17.500919 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c96e3e76-d237-4b63-b2f2-9727a2029078 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '349292c48f0c4f4195e8970f9137de77'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:17.501541 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c96e3e76-d237-4b63-b2f2-9727a2029078 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '349292c48f0c4f4195e8970f9137de77'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:17.518109 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:17.521783 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1136/2272] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:08:17 2022] DELETE /identity/v3/users/349292c48f0c4f4195e8970f9137de77 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:17.526195 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90bf4e-e596-4ed8-84fa-dac29757e680 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:17.526479 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90bf4e-e596-4ed8-84fa-dac29757e680 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:17.526760 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90bf4e-e596-4ed8-84fa-dac29757e680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:17.546209 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-9f90bf4e-e596-4ed8-84fa-dac29757e680 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:17.547467 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-9f90bf4e-e596-4ed8-84fa-dac29757e680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:17.606137 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1137/2273] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:17.616182 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e3e76-d237-4b63-b2f2-9727a2029078 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:17.617655 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c96e3e76-d237-4b63-b2f2-9727a2029078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:17.676295 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e3e76-d237-4b63-b2f2-9727a2029078 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:17.681951 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff558943-9b50-4f31-bda3-57a37f4708f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:17.682626 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff558943-9b50-4f31-bda3-57a37f4708f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:17.682908 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff558943-9b50-4f31-bda3-57a37f4708f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:17.683237 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff558943-9b50-4f31-bda3-57a37f4708f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:17.693576 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff558943-9b50-4f31-bda3-57a37f4708f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:17.696604 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff558943-9b50-4f31-bda3-57a37f4708f6 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:17.719477 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1137/2274] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:08:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:08:17.859019 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:18.142435 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f90bf4e-e596-4ed8-84fa-dac29757e680 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:18.143879 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-9f90bf4e-e596-4ed8-84fa-dac29757e680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:18.156638 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f90bf4e-e596-4ed8-84fa-dac29757e680 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:18.160925 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a7fe5-4c60-49af-9440-8befed014afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:18.161553 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a7fe5-4c60-49af-9440-8befed014afe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:18.161834 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a7fe5-4c60-49af-9440-8befed014afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:18.162106 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a7fe5-4c60-49af-9440-8befed014afe admin admin] PATH_INFO: `/v3/projects/a2242b4c450d4a0cb161a78483fd9c93` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:18.163253 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a7fe5-4c60-49af-9440-8befed014afe admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2242b4c450d4a0cb161a78483fd9c93)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:18.165206 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a7fe5-4c60-49af-9440-8befed014afe admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:18.278739 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a33a7fe5-4c60-49af-9440-8befed014afe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2242b4c450d4a0cb161a78483fd9c93', 'request_id': 'req-a33a7fe5-4c60-49af-9440-8befed014afe'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:18.284268 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:18.352601 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:18.383014 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a33a7fe5-4c60-49af-9440-8befed014afe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2242b4c450d4a0cb161a78483fd9c93', 'request_id': 'req-a33a7fe5-4c60-49af-9440-8befed014afe'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:18.399290 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a33a7fe5-4c60-49af-9440-8befed014afe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2242b4c450d4a0cb161a78483fd9c93', 'request_id': 'req-a33a7fe5-4c60-49af-9440-8befed014afe'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:18.404488 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a33a7fe5-4c60-49af-9440-8befed014afe admin admin] The token cache is being invalidate because project a2242b4c450d4a0cb161a78483fd9c93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:08:18.404488 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a33a7fe5-4c60-49af-9440-8befed014afe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:18.404488 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a33a7fe5-4c60-49af-9440-8befed014afe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:18.408190 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1138/2275] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:08:18 2022] DELETE /identity/v3/projects/a2242b4c450d4a0cb161a78483fd9c93 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:18.413166 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff558943-9b50-4f31-bda3-57a37f4708f6 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:18.423021 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff558943-9b50-4f31-bda3-57a37f4708f6 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:18.481507 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ae391-cb2c-4c15-a652-e4e943d660ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:18.482317 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ae391-cb2c-4c15-a652-e4e943d660ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:18.482597 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ae391-cb2c-4c15-a652-e4e943d660ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:18.482868 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ae391-cb2c-4c15-a652-e4e943d660ec admin admin] PATH_INFO: `/v3/users/3e2c7194598e4ab7ba4e7ed648f8471b` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:18.484074 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ae391-cb2c-4c15-a652-e4e943d660ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e2c7194598e4ab7ba4e7ed648f8471b)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:18.486058 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ae391-cb2c-4c15-a652-e4e943d660ec admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:18.721081 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b99ae391-cb2c-4c15-a652-e4e943d660ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e2c7194598e4ab7ba4e7ed648f8471b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:18.721812 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b99ae391-cb2c-4c15-a652-e4e943d660ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e2c7194598e4ab7ba4e7ed648f8471b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:18.729232 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b99ae391-cb2c-4c15-a652-e4e943d660ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e2c7194598e4ab7ba4e7ed648f8471b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:18.831903 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b99ae391-cb2c-4c15-a652-e4e943d660ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e2c7194598e4ab7ba4e7ed648f8471b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:18.832606 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b99ae391-cb2c-4c15-a652-e4e943d660ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e2c7194598e4ab7ba4e7ed648f8471b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:18.850586 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1138/2276] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:08:18 2022] DELETE /identity/v3/users/3e2c7194598e4ab7ba4e7ed648f8471b => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:19.106866 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:19.296427 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a7fe5-4c60-49af-9440-8befed014afe admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:19.299939 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a33a7fe5-4c60-49af-9440-8befed014afe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:19.311677 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a7fe5-4c60-49af-9440-8befed014afe admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:19.316182 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:19.382239 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b3333-c5d2-425f-afc8-4d0338a0a33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:19.383150 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b3333-c5d2-425f-afc8-4d0338a0a33f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:19.383495 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b3333-c5d2-425f-afc8-4d0338a0a33f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:19.383776 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b3333-c5d2-425f-afc8-4d0338a0a33f admin admin] PATH_INFO: `/v3/projects/dee2f21c9543451eae265749755c518a` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:19.384949 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b3333-c5d2-425f-afc8-4d0338a0a33f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dee2f21c9543451eae265749755c518a)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:19.387475 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b3333-c5d2-425f-afc8-4d0338a0a33f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:19.563103 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-213b3333-c5d2-425f-afc8-4d0338a0a33f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dee2f21c9543451eae265749755c518a', 'request_id': 'req-213b3333-c5d2-425f-afc8-4d0338a0a33f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:19.631706 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-213b3333-c5d2-425f-afc8-4d0338a0a33f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dee2f21c9543451eae265749755c518a', 'request_id': 'req-213b3333-c5d2-425f-afc8-4d0338a0a33f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:19.644106 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-213b3333-c5d2-425f-afc8-4d0338a0a33f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dee2f21c9543451eae265749755c518a', 'request_id': 'req-213b3333-c5d2-425f-afc8-4d0338a0a33f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:19.653104 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-213b3333-c5d2-425f-afc8-4d0338a0a33f admin admin] The token cache is being invalidate because project dee2f21c9543451eae265749755c518a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:08:19.653104 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-213b3333-c5d2-425f-afc8-4d0338a0a33f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:19.653671 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-213b3333-c5d2-425f-afc8-4d0338a0a33f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:19.655257 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1139/2277] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:08:19 2022] DELETE /identity/v3/projects/dee2f21c9543451eae265749755c518a => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:08:19.660188 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a9b63-d3c9-4705-88d2-1cd394e7d074 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:19.661389 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a9b63-d3c9-4705-88d2-1cd394e7d074 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:19.661607 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a9b63-d3c9-4705-88d2-1cd394e7d074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:19.687319 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-8b0a9b63-d3c9-4705-88d2-1cd394e7d074 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:19.687319 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8b0a9b63-d3c9-4705-88d2-1cd394e7d074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:19.744976 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1139/2278] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:19.755593 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98bb36c-72da-43bf-8899-eb607b209be4 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:19.755930 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98bb36c-72da-43bf-8899-eb607b209be4 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:19.756213 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98bb36c-72da-43bf-8899-eb607b209be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:19.778633 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-d98bb36c-72da-43bf-8899-eb607b209be4 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:19.779902 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d98bb36c-72da-43bf-8899-eb607b209be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:19.792117 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1140/2279] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:19.797198 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a9b63-d3c9-4705-88d2-1cd394e7d074 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:19.803759 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a9b63-d3c9-4705-88d2-1cd394e7d074 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:19.808883 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aec36d-ed3d-4cb0-979f-1c84471fca92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:19.809428 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aec36d-ed3d-4cb0-979f-1c84471fca92 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:19.809694 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aec36d-ed3d-4cb0-979f-1c84471fca92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:19.809962 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aec36d-ed3d-4cb0-979f-1c84471fca92 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:19.810602 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aec36d-ed3d-4cb0-979f-1c84471fca92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:08:19.810903 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aec36d-ed3d-4cb0-979f-1c84471fca92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:19.812736 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aec36d-ed3d-4cb0-979f-1c84471fca92 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:19.826135 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1140/2280] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:08:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:08:19.835339 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98bb36c-72da-43bf-8899-eb607b209be4 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:19.845135 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98bb36c-72da-43bf-8899-eb607b209be4 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:19.851104 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aba7dd-b50e-4933-a133-4a0afcfd55bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:19.852023 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aba7dd-b50e-4933-a133-4a0afcfd55bc admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:19.852538 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aba7dd-b50e-4933-a133-4a0afcfd55bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:19.853806 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aba7dd-b50e-4933-a133-4a0afcfd55bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:19.854817 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aba7dd-b50e-4933-a133-4a0afcfd55bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:19.856905 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aba7dd-b50e-4933-a133-4a0afcfd55bc admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:19.994686 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1141/2281] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:08:19 2022] POST /identity/v3/projects => generated 369 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:08:19.998627 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aec36d-ed3d-4cb0-979f-1c84471fca92 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:20.009083 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aec36d-ed3d-4cb0-979f-1c84471fca92 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:20.070282 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:20.070976 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db953b6f-f3c8-4d6f-802f-50b605743c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:20.071585 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db953b6f-f3c8-4d6f-802f-50b605743c1c admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:20.071868 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db953b6f-f3c8-4d6f-802f-50b605743c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:20.072140 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db953b6f-f3c8-4d6f-802f-50b605743c1c admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:20.072830 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db953b6f-f3c8-4d6f-802f-50b605743c1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:20.074790 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db953b6f-f3c8-4d6f-802f-50b605743c1c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:20.188158 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1141/2282] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:08:19 2022] POST /identity/v3/users => generated 285 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:08:20.201946 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aba7dd-b50e-4933-a133-4a0afcfd55bc admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:20.213414 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aba7dd-b50e-4933-a133-4a0afcfd55bc admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:20.218827 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449855f-eddd-4746-9e66-d4ac085f5ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:20.226020 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449855f-eddd-4746-9e66-d4ac085f5ca6 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:20.226372 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449855f-eddd-4746-9e66-d4ac085f5ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:20.226572 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449855f-eddd-4746-9e66-d4ac085f5ca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:20.227235 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b449855f-eddd-4746-9e66-d4ac085f5ca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:20.234578 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b449855f-eddd-4746-9e66-d4ac085f5ca6 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:20.241602 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:20.250039 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1142/2283] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:20 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:08:20.251459 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db953b6f-f3c8-4d6f-802f-50b605743c1c admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:20.271719 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db953b6f-f3c8-4d6f-802f-50b605743c1c admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:20.273232 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b94051-12e6-4bf9-91a5-297fdec3bfe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:20.273921 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b94051-12e6-4bf9-91a5-297fdec3bfe1 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:20.274304 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b94051-12e6-4bf9-91a5-297fdec3bfe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:20.274644 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b94051-12e6-4bf9-91a5-297fdec3bfe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:20.275502 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b94051-12e6-4bf9-91a5-297fdec3bfe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:20.277234 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b94051-12e6-4bf9-91a5-297fdec3bfe1 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:20.289048 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1142/2284] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 01:08:20.293895 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449855f-eddd-4746-9e66-d4ac085f5ca6 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:20.305610 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449855f-eddd-4746-9e66-d4ac085f5ca6 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:20.310357 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f02cd-3172-47e4-8a3e-8b9778e470b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:20.311008 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f02cd-3172-47e4-8a3e-8b9778e470b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:20.311288 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f02cd-3172-47e4-8a3e-8b9778e470b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:20.311561 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f02cd-3172-47e4-8a3e-8b9778e470b7 admin admin] PATH_INFO: `/v3/projects/e7f5cfc982414991a24dadaea55c21bc/users/51c33d0bba03455a82cfc51cc2a8f657/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:20.327975 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f02cd-3172-47e4-8a3e-8b9778e470b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7f5cfc982414991a24dadaea55c21bc, user_id=51c33d0bba03455a82cfc51cc2a8f657, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:20.330836 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f02cd-3172-47e4-8a3e-8b9778e470b7 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:20.425181 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1143/2285] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:08:20 2022] PUT /identity/v3/projects/e7f5cfc982414991a24dadaea55c21bc/users/51c33d0bba03455a82cfc51cc2a8f657/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:20.435842 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16357f2-36ac-4b41-a544-be8c22616692 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:20.436123 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16357f2-36ac-4b41-a544-be8c22616692 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:20.436400 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16357f2-36ac-4b41-a544-be8c22616692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:20.465193 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f16357f2-36ac-4b41-a544-be8c22616692 None None] MFA Rules not processed for user `51c33d0bba03455a82cfc51cc2a8f657`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:20.466812 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f16357f2-36ac-4b41-a544-be8c22616692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:20.539876 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1143/2286] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:20.544410 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f02cd-3172-47e4-8a3e-8b9778e470b7 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:20.614718 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f02cd-3172-47e4-8a3e-8b9778e470b7 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:20.621295 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79f4cd-18de-4f16-b267-73eb290d8a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:20.621920 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79f4cd-18de-4f16-b267-73eb290d8a46 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:20.622547 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79f4cd-18de-4f16-b267-73eb290d8a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:20.622832 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79f4cd-18de-4f16-b267-73eb290d8a46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:20.623473 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f79f4cd-18de-4f16-b267-73eb290d8a46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:20.626094 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f79f4cd-18de-4f16-b267-73eb290d8a46 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:20.653228 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:20.734181 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1144/2287] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:08:20 2022] POST /identity/v3/projects => generated 367 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:08:20.734181 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:20.735323 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16357f2-36ac-4b41-a544-be8c22616692 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:20.746760 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16357f2-36ac-4b41-a544-be8c22616692 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:20.814925 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76737c-c41f-48f4-af0c-f589707ee842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:20.815581 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76737c-c41f-48f4-af0c-f589707ee842 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:20.815868 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76737c-c41f-48f4-af0c-f589707ee842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:20.816146 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76737c-c41f-48f4-af0c-f589707ee842 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:20.817391 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76737c-c41f-48f4-af0c-f589707ee842 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:20.819355 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76737c-c41f-48f4-af0c-f589707ee842 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:20.864985 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:20.904290 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1144/2288] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:08:20 2022] POST /identity/v3/users => generated 290 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:08:20.908357 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79f4cd-18de-4f16-b267-73eb290d8a46 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:20.919464 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79f4cd-18de-4f16-b267-73eb290d8a46 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:20.924216 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee02253-1592-4c47-aafd-fa2be81bbc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:20.925107 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee02253-1592-4c47-aafd-fa2be81bbc4d admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:20.925397 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee02253-1592-4c47-aafd-fa2be81bbc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:20.925677 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee02253-1592-4c47-aafd-fa2be81bbc4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:20.926907 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee02253-1592-4c47-aafd-fa2be81bbc4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:20.928586 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee02253-1592-4c47-aafd-fa2be81bbc4d admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:20.940396 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1145/2289] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:08:20.945044 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76737c-c41f-48f4-af0c-f589707ee842 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:20.955380 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76737c-c41f-48f4-af0c-f589707ee842 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:20.961765 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87ca17-3221-44e9-aa56-33b393186482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:20.962471 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87ca17-3221-44e9-aa56-33b393186482 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:20.962852 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87ca17-3221-44e9-aa56-33b393186482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:20.963136 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87ca17-3221-44e9-aa56-33b393186482 admin admin] PATH_INFO: `/v3/projects/e1be28e0feae49f0aaf767af17058024/users/a9191e3204314c61b60298cbad66218c/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:20.980388 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa87ca17-3221-44e9-aa56-33b393186482 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1be28e0feae49f0aaf767af17058024, user_id=a9191e3204314c61b60298cbad66218c, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:20.983308 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa87ca17-3221-44e9-aa56-33b393186482 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:20.992382 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:21.077169 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1145/2290] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:08:20 2022] PUT /identity/v3/projects/e1be28e0feae49f0aaf767af17058024/users/a9191e3204314c61b60298cbad66218c/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:21.078540 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee02253-1592-4c47-aafd-fa2be81bbc4d admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:21.089446 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee02253-1592-4c47-aafd-fa2be81bbc4d admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:21.150129 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:21.153704 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ddf77b-c5a1-447d-8746-6e88aacbb5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:21.154339 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ddf77b-c5a1-447d-8746-6e88aacbb5c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:21.154650 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ddf77b-c5a1-447d-8746-6e88aacbb5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:21.154927 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ddf77b-c5a1-447d-8746-6e88aacbb5c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:21.155577 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ddf77b-c5a1-447d-8746-6e88aacbb5c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:21.157287 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ddf77b-c5a1-447d-8746-6e88aacbb5c1 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:21.169251 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1146/2291] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:21 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:08:21.171755 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87ca17-3221-44e9-aa56-33b393186482 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:21.182504 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87ca17-3221-44e9-aa56-33b393186482 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:21.186968 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab7c2a-c39d-4431-babf-2bf380676129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:21.187618 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab7c2a-c39d-4431-babf-2bf380676129 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:21.187901 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab7c2a-c39d-4431-babf-2bf380676129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:21.188175 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab7c2a-c39d-4431-babf-2bf380676129 admin admin] PATH_INFO: `/v3/projects/e1be28e0feae49f0aaf767af17058024/users/a9191e3204314c61b60298cbad66218c/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:21.190196 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab7c2a-c39d-4431-babf-2bf380676129 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1be28e0feae49f0aaf767af17058024, user_id=a9191e3204314c61b60298cbad66218c, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:21.193529 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab7c2a-c39d-4431-babf-2bf380676129 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:21.203912 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1146/2292] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:08:21 2022] PUT /identity/v3/projects/e1be28e0feae49f0aaf767af17058024/users/a9191e3204314c61b60298cbad66218c/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:08:21.214130 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0f00f-21f8-43a9-bbca-8b56b1eea555 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:21.214590 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0f00f-21f8-43a9-bbca-8b56b1eea555 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:21.214681 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0f00f-21f8-43a9-bbca-8b56b1eea555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:21.230615 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:21.246227 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-f8f0f00f-21f8-43a9-bbca-8b56b1eea555 None None] MFA Rules not processed for user `a9191e3204314c61b60298cbad66218c`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:21.247446 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f8f0f00f-21f8-43a9-bbca-8b56b1eea555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:21.288433 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab7c2a-c39d-4431-babf-2bf380676129 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:21.290598 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a9ab7c2a-c39d-4431-babf-2bf380676129 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:21.304615 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:21.326300 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1147/2293] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:21 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:21.337689 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f0f00f-21f8-43a9-bbca-8b56b1eea555 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:21.339205 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f8f0f00f-21f8-43a9-bbca-8b56b1eea555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:21.358056 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab7c2a-c39d-4431-babf-2bf380676129 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:21.363064 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e736e-09dc-47ae-a818-f94aba932dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:21.364282 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e736e-09dc-47ae-a818-f94aba932dfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:21.364761 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e736e-09dc-47ae-a818-f94aba932dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:21.365042 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e736e-09dc-47ae-a818-f94aba932dfb admin admin] PATH_INFO: `/v3/users/e1b78df6534141d29020d384ddf311b6` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:21.366288 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464e736e-09dc-47ae-a818-f94aba932dfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1b78df6534141d29020d384ddf311b6)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:21.368209 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464e736e-09dc-47ae-a818-f94aba932dfb admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:21.401372 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f0f00f-21f8-43a9-bbca-8b56b1eea555 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:21.405168 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9a5a1-e12b-4851-9e9f-fb26cda106ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:21.405782 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9a5a1-e12b-4851-9e9f-fb26cda106ab service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:21.406058 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9a5a1-e12b-4851-9e9f-fb26cda106ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:21.406327 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9a5a1-e12b-4851-9e9f-fb26cda106ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:21.417029 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e9a5a1-e12b-4851-9e9f-fb26cda106ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:21.418504 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e9a5a1-e12b-4851-9e9f-fb26cda106ab service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:21.429228 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1148/2294] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:08:21 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 01:08:21.530047 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:21.545369 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9a5a1-e12b-4851-9e9f-fb26cda106ab service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:21.548170 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-86e9a5a1-e12b-4851-9e9f-fb26cda106ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:21.558417 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-464e736e-09dc-47ae-a818-f94aba932dfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1b78df6534141d29020d384ddf311b6'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:21.559107 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-464e736e-09dc-47ae-a818-f94aba932dfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1b78df6534141d29020d384ddf311b6'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:21.564534 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9a5a1-e12b-4851-9e9f-fb26cda106ab service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:21.568983 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-464e736e-09dc-47ae-a818-f94aba932dfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1b78df6534141d29020d384ddf311b6'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:21.571495 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb99c557-70dc-4fc9-907a-7b3c4cb21acf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:21.572333 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99c557-70dc-4fc9-907a-7b3c4cb21acf service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:21.572333 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99c557-70dc-4fc9-907a-7b3c4cb21acf service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:21.572652 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99c557-70dc-4fc9-907a-7b3c4cb21acf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:21.575290 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-cb99c557-70dc-4fc9-907a-7b3c4cb21acf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:21.620416 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-464e736e-09dc-47ae-a818-f94aba932dfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1b78df6534141d29020d384ddf311b6'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:21.621422 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-464e736e-09dc-47ae-a818-f94aba932dfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1b78df6534141d29020d384ddf311b6'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:21.639182 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb99c557-70dc-4fc9-907a-7b3c4cb21acf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:21.640675 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb99c557-70dc-4fc9-907a-7b3c4cb21acf service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:21.641790 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1147/2295] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:08:21 2022] DELETE /identity/v3/users/e1b78df6534141d29020d384ddf311b6 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:08:21.646361 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5f1bc-36e1-4e95-9788-c46f36aae664 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:21.646765 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5f1bc-36e1-4e95-9788-c46f36aae664 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:21.647738 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5f1bc-36e1-4e95-9788-c46f36aae664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:21.650806 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1149/2296] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:08:21 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 01:08:21.668649 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-26c5f1bc-36e1-4e95-9788-c46f36aae664 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:21.669838 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-26c5f1bc-36e1-4e95-9788-c46f36aae664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:21.675653 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:21.716253 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:21.731372 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1148/2297] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:08:21.742440 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb99c557-70dc-4fc9-907a-7b3c4cb21acf service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:21.743990 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-cb99c557-70dc-4fc9-907a-7b3c4cb21acf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:21.806259 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb99c557-70dc-4fc9-907a-7b3c4cb21acf service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:21.809877 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:21.859159 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635268a6-4952-43f4-bae1-87ebd81f9f0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:21.860024 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635268a6-4952-43f4-bae1-87ebd81f9f0b service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:21.860310 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635268a6-4952-43f4-bae1-87ebd81f9f0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:21.860621 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635268a6-4952-43f4-bae1-87ebd81f9f0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:21.873732 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635268a6-4952-43f4-bae1-87ebd81f9f0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:21.876087 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635268a6-4952-43f4-bae1-87ebd81f9f0b service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:21.887986 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1150/2298] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:08:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:08:22.541760 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c5f1bc-36e1-4e95-9788-c46f36aae664 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:22.549751 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-26c5f1bc-36e1-4e95-9788-c46f36aae664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:22.574053 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c5f1bc-36e1-4e95-9788-c46f36aae664 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:22.580876 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90035b71-9d37-4c92-86bf-e3fffa25250e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:22.584348 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90035b71-9d37-4c92-86bf-e3fffa25250e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:22.584348 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90035b71-9d37-4c92-86bf-e3fffa25250e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:22.584348 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90035b71-9d37-4c92-86bf-e3fffa25250e admin admin] PATH_INFO: `/v3/projects/b14ba9e7b3524b509ef833c2c9bd8067` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:22.584986 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90035b71-9d37-4c92-86bf-e3fffa25250e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b14ba9e7b3524b509ef833c2c9bd8067)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:22.587615 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90035b71-9d37-4c92-86bf-e3fffa25250e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:22.727700 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-90035b71-9d37-4c92-86bf-e3fffa25250e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b14ba9e7b3524b509ef833c2c9bd8067', 'request_id': 'req-90035b71-9d37-4c92-86bf-e3fffa25250e'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:22.823892 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-90035b71-9d37-4c92-86bf-e3fffa25250e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b14ba9e7b3524b509ef833c2c9bd8067', 'request_id': 'req-90035b71-9d37-4c92-86bf-e3fffa25250e'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:22.830129 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-90035b71-9d37-4c92-86bf-e3fffa25250e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b14ba9e7b3524b509ef833c2c9bd8067', 'request_id': 'req-90035b71-9d37-4c92-86bf-e3fffa25250e'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:22.842110 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-90035b71-9d37-4c92-86bf-e3fffa25250e admin admin] The token cache is being invalidate because project b14ba9e7b3524b509ef833c2c9bd8067 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:08:22.842401 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-90035b71-9d37-4c92-86bf-e3fffa25250e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:22.843037 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-90035b71-9d37-4c92-86bf-e3fffa25250e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:22.844625 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1149/2299] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:08:22 2022] DELETE /identity/v3/projects/b14ba9e7b3524b509ef833c2c9bd8067 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:22.850114 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceffb2d-31aa-4b2b-bf2d-5717127eee56 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:22.851135 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceffb2d-31aa-4b2b-bf2d-5717127eee56 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:22.851135 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceffb2d-31aa-4b2b-bf2d-5717127eee56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:22.871546 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-3ceffb2d-31aa-4b2b-bf2d-5717127eee56 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:22.872789 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3ceffb2d-31aa-4b2b-bf2d-5717127eee56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:22.932762 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1151/2300] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:22.940726 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5b9b3-6fba-41f1-a392-a049dc890be4 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:22.941008 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5b9b3-6fba-41f1-a392-a049dc890be4 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:22.941288 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5b9b3-6fba-41f1-a392-a049dc890be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:22.964180 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-8cb5b9b3-6fba-41f1-a392-a049dc890be4 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:22.965536 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8cb5b9b3-6fba-41f1-a392-a049dc890be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:22.973363 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1150/2301] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:08:22.977822 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceffb2d-31aa-4b2b-bf2d-5717127eee56 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:22.988276 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceffb2d-31aa-4b2b-bf2d-5717127eee56 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:22.993503 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34a9097-2727-48cb-8e08-8f6c0e043c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:22.993907 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a9097-2727-48cb-8e08-8f6c0e043c98 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:22.994186 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a9097-2727-48cb-8e08-8f6c0e043c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:22.994457 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a9097-2727-48cb-8e08-8f6c0e043c98 admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:22.995170 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34a9097-2727-48cb-8e08-8f6c0e043c98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:08:22.995397 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34a9097-2727-48cb-8e08-8f6c0e043c98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:22.997494 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34a9097-2727-48cb-8e08-8f6c0e043c98 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:23.013121 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1152/2302] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:08:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:08:23.017464 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb5b9b3-6fba-41f1-a392-a049dc890be4 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:23.027411 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb5b9b3-6fba-41f1-a392-a049dc890be4 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:23.032074 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f06517-58bf-4e2c-9df4-2797ddfe1df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:23.032696 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f06517-58bf-4e2c-9df4-2797ddfe1df7 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:23.032972 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f06517-58bf-4e2c-9df4-2797ddfe1df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:23.033241 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f06517-58bf-4e2c-9df4-2797ddfe1df7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:23.033873 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f06517-58bf-4e2c-9df4-2797ddfe1df7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:23.035890 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f06517-58bf-4e2c-9df4-2797ddfe1df7 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:23.151516 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1151/2303] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:08:23 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:08:23.155331 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34a9097-2727-48cb-8e08-8f6c0e043c98 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:23.173598 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34a9097-2727-48cb-8e08-8f6c0e043c98 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:23.239386 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4593a6-a26a-4aab-9cd4-bbbde5831dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:23.239702 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4593a6-a26a-4aab-9cd4-bbbde5831dd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:23.240043 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4593a6-a26a-4aab-9cd4-bbbde5831dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:23.240400 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4593a6-a26a-4aab-9cd4-bbbde5831dd7 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:23.241183 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4593a6-a26a-4aab-9cd4-bbbde5831dd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:23.243082 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4593a6-a26a-4aab-9cd4-bbbde5831dd7 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:23.338869 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1153/2304] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:08:23 2022] POST /identity/v3/users => generated 287 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:08:23.343339 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f06517-58bf-4e2c-9df4-2797ddfe1df7 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:23.355369 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f06517-58bf-4e2c-9df4-2797ddfe1df7 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:23.360623 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84430131-d0c0-44be-bf52-3bd082998718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:23.364989 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84430131-d0c0-44be-bf52-3bd082998718 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:23.364989 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84430131-d0c0-44be-bf52-3bd082998718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:23.364989 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84430131-d0c0-44be-bf52-3bd082998718 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:23.364989 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84430131-d0c0-44be-bf52-3bd082998718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:23.364989 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84430131-d0c0-44be-bf52-3bd082998718 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:23.375557 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1152/2305] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:08:23.380151 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4593a6-a26a-4aab-9cd4-bbbde5831dd7 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:23.390988 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4593a6-a26a-4aab-9cd4-bbbde5831dd7 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:23.396386 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44f9c8-26dc-46b4-a28f-76a8bdfd217c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:23.397016 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44f9c8-26dc-46b4-a28f-76a8bdfd217c admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:23.397447 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44f9c8-26dc-46b4-a28f-76a8bdfd217c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:23.397720 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44f9c8-26dc-46b4-a28f-76a8bdfd217c admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:23.398354 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e44f9c8-26dc-46b4-a28f-76a8bdfd217c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:23.399999 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e44f9c8-26dc-46b4-a28f-76a8bdfd217c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:23.411532 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1154/2306] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:08:23.416325 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84430131-d0c0-44be-bf52-3bd082998718 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:23.427295 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84430131-d0c0-44be-bf52-3bd082998718 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:23.431757 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99094c71-7c75-4f5d-92e5-a377837f25df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:23.432406 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99094c71-7c75-4f5d-92e5-a377837f25df admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:23.432724 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99094c71-7c75-4f5d-92e5-a377837f25df admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:23.433003 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99094c71-7c75-4f5d-92e5-a377837f25df admin admin] PATH_INFO: `/v3/projects/29badb5db89f4bc7b2b2dc953f600a0d/users/4a11e0e4c6ef48c29ba67dbd9e7affd2/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:23.451323 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99094c71-7c75-4f5d-92e5-a377837f25df admin admin] RBAC: Authorizing `identity:create_grant(project_id=29badb5db89f4bc7b2b2dc953f600a0d, user_id=4a11e0e4c6ef48c29ba67dbd9e7affd2, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:23.451842 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99094c71-7c75-4f5d-92e5-a377837f25df admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:23.540637 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1153/2307] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:08:23 2022] PUT /identity/v3/projects/29badb5db89f4bc7b2b2dc953f600a0d/users/4a11e0e4c6ef48c29ba67dbd9e7affd2/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:23.552693 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba37e2-bf91-4f6c-aa93-486871189e9f None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:23.552989 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba37e2-bf91-4f6c-aa93-486871189e9f None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:23.553271 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba37e2-bf91-4f6c-aa93-486871189e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:23.575877 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:23.582864 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-bbba37e2-bf91-4f6c-aa93-486871189e9f None None] MFA Rules not processed for user `4a11e0e4c6ef48c29ba67dbd9e7affd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:23.584194 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-bbba37e2-bf91-4f6c-aa93-486871189e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:23.651683 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1155/2308] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:23.669072 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99094c71-7c75-4f5d-92e5-a377837f25df admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:23.669072 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-99094c71-7c75-4f5d-92e5-a377837f25df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:23.725712 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99094c71-7c75-4f5d-92e5-a377837f25df admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:23.729390 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d01dbc-a880-42d2-b56d-a375de767292 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:23.729988 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d01dbc-a880-42d2-b56d-a375de767292 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:23.730269 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d01dbc-a880-42d2-b56d-a375de767292 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:23.730559 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d01dbc-a880-42d2-b56d-a375de767292 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:23.738763 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d01dbc-a880-42d2-b56d-a375de767292 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:23.740782 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d01dbc-a880-42d2-b56d-a375de767292 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:23.749650 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1154/2309] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:08:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:08:23.775975 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbba37e2-bf91-4f6c-aa93-486871189e9f None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:23.840920 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbba37e2-bf91-4f6c-aa93-486871189e9f None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:23.845774 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d073a0-112d-46a7-a1d7-d5890a5a4953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:23.846419 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d073a0-112d-46a7-a1d7-d5890a5a4953 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:23.846705 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d073a0-112d-46a7-a1d7-d5890a5a4953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:23.846983 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d073a0-112d-46a7-a1d7-d5890a5a4953 admin admin] PATH_INFO: `/v3/users/4a11e0e4c6ef48c29ba67dbd9e7affd2` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:23.848758 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d073a0-112d-46a7-a1d7-d5890a5a4953 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a11e0e4c6ef48c29ba67dbd9e7affd2)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:23.850816 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d073a0-112d-46a7-a1d7-d5890a5a4953 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:24.044707 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e8d073a0-112d-46a7-a1d7-d5890a5a4953 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a11e0e4c6ef48c29ba67dbd9e7affd2'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:24.045369 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e8d073a0-112d-46a7-a1d7-d5890a5a4953 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a11e0e4c6ef48c29ba67dbd9e7affd2'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:24.115725 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e8d073a0-112d-46a7-a1d7-d5890a5a4953 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a11e0e4c6ef48c29ba67dbd9e7affd2'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:24.116362 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e8d073a0-112d-46a7-a1d7-d5890a5a4953 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a11e0e4c6ef48c29ba67dbd9e7affd2'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:24.128685 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e8d073a0-112d-46a7-a1d7-d5890a5a4953 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a11e0e4c6ef48c29ba67dbd9e7affd2'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:24.141398 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1156/2310] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:08:23 2022] DELETE /identity/v3/users/4a11e0e4c6ef48c29ba67dbd9e7affd2 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:24.146733 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0c92a-c215-4a75-b982-d37a502a047b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:24.147020 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0c92a-c215-4a75-b982-d37a502a047b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:24.147293 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0c92a-c215-4a75-b982-d37a502a047b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:24.166810 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-fbb0c92a-c215-4a75-b982-d37a502a047b None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:24.168000 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-fbb0c92a-c215-4a75-b982-d37a502a047b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:24.173269 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:24.251608 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1155/2311] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:08:24.261183 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d073a0-112d-46a7-a1d7-d5890a5a4953 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:24.265651 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e8d073a0-112d-46a7-a1d7-d5890a5a4953 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:24.294389 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:24.343861 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d073a0-112d-46a7-a1d7-d5890a5a4953 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:24.347534 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc9efef-86b1-4ef0-890b-35188f17176d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:24.348346 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9efef-86b1-4ef0-890b-35188f17176d service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:24.349953 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9efef-86b1-4ef0-890b-35188f17176d service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:24.350597 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9efef-86b1-4ef0-890b-35188f17176d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:24.361116 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc9efef-86b1-4ef0-890b-35188f17176d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:24.362951 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc9efef-86b1-4ef0-890b-35188f17176d service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:24.388864 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1157/2312] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:08:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:08:24.779831 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0c92a-c215-4a75-b982-d37a502a047b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:24.782465 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-fbb0c92a-c215-4a75-b982-d37a502a047b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:24.796856 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0c92a-c215-4a75-b982-d37a502a047b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:24.802331 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6f84e4-c952-40db-8d2f-72ab9a086064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:24.802978 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f84e4-c952-40db-8d2f-72ab9a086064 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:24.805848 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f84e4-c952-40db-8d2f-72ab9a086064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:24.805848 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f84e4-c952-40db-8d2f-72ab9a086064 admin admin] PATH_INFO: `/v3/projects/29badb5db89f4bc7b2b2dc953f600a0d` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:24.805848 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6f84e4-c952-40db-8d2f-72ab9a086064 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29badb5db89f4bc7b2b2dc953f600a0d)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:24.806902 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6f84e4-c952-40db-8d2f-72ab9a086064 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:24.923068 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1e6f84e4-c952-40db-8d2f-72ab9a086064 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29badb5db89f4bc7b2b2dc953f600a0d', 'request_id': 'req-1e6f84e4-c952-40db-8d2f-72ab9a086064'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:24.995920 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1e6f84e4-c952-40db-8d2f-72ab9a086064 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29badb5db89f4bc7b2b2dc953f600a0d', 'request_id': 'req-1e6f84e4-c952-40db-8d2f-72ab9a086064'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:25.003246 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1e6f84e4-c952-40db-8d2f-72ab9a086064 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29badb5db89f4bc7b2b2dc953f600a0d', 'request_id': 'req-1e6f84e4-c952-40db-8d2f-72ab9a086064'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:25.024319 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1e6f84e4-c952-40db-8d2f-72ab9a086064 admin admin] The token cache is being invalidate because project 29badb5db89f4bc7b2b2dc953f600a0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:08:25.024550 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1e6f84e4-c952-40db-8d2f-72ab9a086064 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:25.025228 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1e6f84e4-c952-40db-8d2f-72ab9a086064 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:25.035007 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1156/2313] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:08:24 2022] DELETE /identity/v3/projects/29badb5db89f4bc7b2b2dc953f600a0d => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:25.040072 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ef756-cdbe-4435-935c-56d1e7f86d78 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:25.040567 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ef756-cdbe-4435-935c-56d1e7f86d78 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:25.040660 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ef756-cdbe-4435-935c-56d1e7f86d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:25.065249 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-120ef756-cdbe-4435-935c-56d1e7f86d78 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:25.066480 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-120ef756-cdbe-4435-935c-56d1e7f86d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:25.128703 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1158/2314] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:08:25.137231 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337dbd1-b207-49eb-a516-9469111b2df4 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:25.137631 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337dbd1-b207-49eb-a516-9469111b2df4 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:25.137908 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337dbd1-b207-49eb-a516-9469111b2df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:25.160041 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-4337dbd1-b207-49eb-a516-9469111b2df4 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:25.161768 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4337dbd1-b207-49eb-a516-9469111b2df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:25.169441 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1157/2315] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:25.173741 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120ef756-cdbe-4435-935c-56d1e7f86d78 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:25.183907 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120ef756-cdbe-4435-935c-56d1e7f86d78 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:25.188365 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa55281-943d-4595-98ff-9419dba6d5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:25.189016 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa55281-943d-4595-98ff-9419dba6d5e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:25.189293 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa55281-943d-4595-98ff-9419dba6d5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:25.189567 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa55281-943d-4595-98ff-9419dba6d5e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:25.190204 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa55281-943d-4595-98ff-9419dba6d5e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:08:25.190504 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa55281-943d-4595-98ff-9419dba6d5e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:25.192214 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa55281-943d-4595-98ff-9419dba6d5e6 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:25.203774 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1159/2316] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:08:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:08:25.207979 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4337dbd1-b207-49eb-a516-9469111b2df4 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:25.217150 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4337dbd1-b207-49eb-a516-9469111b2df4 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:25.221757 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2069983-3a1d-4591-bbba-b05c7e3e018f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:25.222355 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2069983-3a1d-4591-bbba-b05c7e3e018f admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:25.222634 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2069983-3a1d-4591-bbba-b05c7e3e018f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:25.222910 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2069983-3a1d-4591-bbba-b05c7e3e018f admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:25.224057 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2069983-3a1d-4591-bbba-b05c7e3e018f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:25.226004 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2069983-3a1d-4591-bbba-b05c7e3e018f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:25.313402 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1158/2317] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:08:25 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:08:25.317599 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa55281-943d-4595-98ff-9419dba6d5e6 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:25.330542 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa55281-943d-4595-98ff-9419dba6d5e6 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:25.392876 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac69735-0c90-41ec-bef8-22bfccf6ebb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:25.393592 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac69735-0c90-41ec-bef8-22bfccf6ebb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:25.393978 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac69735-0c90-41ec-bef8-22bfccf6ebb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:25.394612 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac69735-0c90-41ec-bef8-22bfccf6ebb7 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:25.394877 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac69735-0c90-41ec-bef8-22bfccf6ebb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:25.396921 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac69735-0c90-41ec-bef8-22bfccf6ebb7 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:25.505561 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1160/2318] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:08:25 2022] POST /identity/v3/users => generated 287 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:08:25.509433 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2069983-3a1d-4591-bbba-b05c7e3e018f admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:25.521174 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2069983-3a1d-4591-bbba-b05c7e3e018f admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:25.526366 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21d87e-1763-4411-87d6-872618fece4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:25.527498 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21d87e-1763-4411-87d6-872618fece4d admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:25.527498 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21d87e-1763-4411-87d6-872618fece4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:25.527498 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21d87e-1763-4411-87d6-872618fece4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:25.528202 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c21d87e-1763-4411-87d6-872618fece4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:25.529892 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c21d87e-1763-4411-87d6-872618fece4d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:25.544660 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1159/2319] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:08:25.545909 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac69735-0c90-41ec-bef8-22bfccf6ebb7 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:25.556601 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac69735-0c90-41ec-bef8-22bfccf6ebb7 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:25.561657 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f837a51-1ff1-4a5f-a1de-db1c4ae1f99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:25.562332 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f837a51-1ff1-4a5f-a1de-db1c4ae1f99f admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:25.562684 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f837a51-1ff1-4a5f-a1de-db1c4ae1f99f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:25.563045 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f837a51-1ff1-4a5f-a1de-db1c4ae1f99f admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:25.563698 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f837a51-1ff1-4a5f-a1de-db1c4ae1f99f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:25.565466 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f837a51-1ff1-4a5f-a1de-db1c4ae1f99f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:25.581288 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1161/2320] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:08:25.583278 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21d87e-1763-4411-87d6-872618fece4d admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:25.595438 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21d87e-1763-4411-87d6-872618fece4d admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:25.601094 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae20ad6-00c1-4c1e-b44d-737fba4a49a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:25.601824 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae20ad6-00c1-4c1e-b44d-737fba4a49a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:25.602182 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae20ad6-00c1-4c1e-b44d-737fba4a49a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:25.602536 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae20ad6-00c1-4c1e-b44d-737fba4a49a6 admin admin] PATH_INFO: `/v3/projects/98424e58111945b28f71e09fb5ad573f/users/cfe0e4123d8845d08d9966391ea54bd2/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:25.620079 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae20ad6-00c1-4c1e-b44d-737fba4a49a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98424e58111945b28f71e09fb5ad573f, user_id=cfe0e4123d8845d08d9966391ea54bd2, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:25.624231 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae20ad6-00c1-4c1e-b44d-737fba4a49a6 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:25.718954 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1160/2321] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:08:25 2022] PUT /identity/v3/projects/98424e58111945b28f71e09fb5ad573f/users/cfe0e4123d8845d08d9966391ea54bd2/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:25.730399 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f656aa-f585-4882-baa6-51480fb514a2 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:25.730710 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f656aa-f585-4882-baa6-51480fb514a2 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:25.731040 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f656aa-f585-4882-baa6-51480fb514a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:25.762643 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-c8f656aa-f585-4882-baa6-51480fb514a2 None None] MFA Rules not processed for user `cfe0e4123d8845d08d9966391ea54bd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:25.763899 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c8f656aa-f585-4882-baa6-51480fb514a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:25.781845 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:25.830705 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1162/2322] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:08:25.843376 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae20ad6-00c1-4c1e-b44d-737fba4a49a6 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:25.846143 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0ae20ad6-00c1-4c1e-b44d-737fba4a49a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:25.886070 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:25.911695 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae20ad6-00c1-4c1e-b44d-737fba4a49a6 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:25.915447 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f297bc-1423-4cb8-aab1-299703063b60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:25.916061 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f297bc-1423-4cb8-aab1-299703063b60 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:25.916336 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f297bc-1423-4cb8-aab1-299703063b60 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:25.916641 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f297bc-1423-4cb8-aab1-299703063b60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:25.925134 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f297bc-1423-4cb8-aab1-299703063b60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:25.927322 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f297bc-1423-4cb8-aab1-299703063b60 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:25.937367 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1161/2323] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:08:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:08:26.906596 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f656aa-f585-4882-baa6-51480fb514a2 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:26.908036 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c8f656aa-f585-4882-baa6-51480fb514a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:27.032628 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f656aa-f585-4882-baa6-51480fb514a2 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:27.040782 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a072f2-4c90-4874-a806-4bbef3f6b8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:27.041814 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a072f2-4c90-4874-a806-4bbef3f6b8ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:27.042114 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a072f2-4c90-4874-a806-4bbef3f6b8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:27.042393 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a072f2-4c90-4874-a806-4bbef3f6b8ca admin admin] PATH_INFO: `/v3/users/51c33d0bba03455a82cfc51cc2a8f657` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:27.043585 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a072f2-4c90-4874-a806-4bbef3f6b8ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=51c33d0bba03455a82cfc51cc2a8f657)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:27.045710 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a072f2-4c90-4874-a806-4bbef3f6b8ca admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:27.275725 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-50a072f2-4c90-4874-a806-4bbef3f6b8ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51c33d0bba03455a82cfc51cc2a8f657'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:27.276742 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-50a072f2-4c90-4874-a806-4bbef3f6b8ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51c33d0bba03455a82cfc51cc2a8f657'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:27.366799 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-50a072f2-4c90-4874-a806-4bbef3f6b8ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51c33d0bba03455a82cfc51cc2a8f657'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:27.367513 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-50a072f2-4c90-4874-a806-4bbef3f6b8ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51c33d0bba03455a82cfc51cc2a8f657'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:27.380128 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-50a072f2-4c90-4874-a806-4bbef3f6b8ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51c33d0bba03455a82cfc51cc2a8f657'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:27.392676 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1163/2324] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:08:26 2022] DELETE /identity/v3/users/51c33d0bba03455a82cfc51cc2a8f657 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:27.397529 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd0c9c3-1e33-4a59-9bf3-7c88b23728e9 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:27.398036 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd0c9c3-1e33-4a59-9bf3-7c88b23728e9 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:27.398036 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd0c9c3-1e33-4a59-9bf3-7c88b23728e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:27.417586 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5bd0c9c3-1e33-4a59-9bf3-7c88b23728e9 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:27.424051 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5bd0c9c3-1e33-4a59-9bf3-7c88b23728e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:27.482578 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1162/2325] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:08:27.493558 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a072f2-4c90-4874-a806-4bbef3f6b8ca admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:27.494962 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-50a072f2-4c90-4874-a806-4bbef3f6b8ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:27.546159 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:27.561847 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a072f2-4c90-4874-a806-4bbef3f6b8ca admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:27.566030 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f85964-b63a-47e4-88ad-6f5b3237ae5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:27.566641 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f85964-b63a-47e4-88ad-6f5b3237ae5e service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:27.566941 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f85964-b63a-47e4-88ad-6f5b3237ae5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:27.567225 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f85964-b63a-47e4-88ad-6f5b3237ae5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:27.576705 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f85964-b63a-47e4-88ad-6f5b3237ae5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:27.578442 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f85964-b63a-47e4-88ad-6f5b3237ae5e service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:27.587788 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1164/2326] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:08:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:08:28.026089 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd0c9c3-1e33-4a59-9bf3-7c88b23728e9 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:28.027568 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5bd0c9c3-1e33-4a59-9bf3-7c88b23728e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:28.040047 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd0c9c3-1e33-4a59-9bf3-7c88b23728e9 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:28.045779 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77862dfc-6d25-432f-869e-98cc8fa1335f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:28.046472 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77862dfc-6d25-432f-869e-98cc8fa1335f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:28.046750 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77862dfc-6d25-432f-869e-98cc8fa1335f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:28.047181 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77862dfc-6d25-432f-869e-98cc8fa1335f admin admin] PATH_INFO: `/v3/projects/e7f5cfc982414991a24dadaea55c21bc` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:28.048338 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77862dfc-6d25-432f-869e-98cc8fa1335f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7f5cfc982414991a24dadaea55c21bc)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:28.050344 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77862dfc-6d25-432f-869e-98cc8fa1335f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:28.147329 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-77862dfc-6d25-432f-869e-98cc8fa1335f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7f5cfc982414991a24dadaea55c21bc', 'request_id': 'req-77862dfc-6d25-432f-869e-98cc8fa1335f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:28.202917 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-77862dfc-6d25-432f-869e-98cc8fa1335f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7f5cfc982414991a24dadaea55c21bc', 'request_id': 'req-77862dfc-6d25-432f-869e-98cc8fa1335f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:28.209616 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-77862dfc-6d25-432f-869e-98cc8fa1335f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7f5cfc982414991a24dadaea55c21bc', 'request_id': 'req-77862dfc-6d25-432f-869e-98cc8fa1335f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:28.222267 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-77862dfc-6d25-432f-869e-98cc8fa1335f admin admin] The token cache is being invalidate because project e7f5cfc982414991a24dadaea55c21bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:08:28.222561 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-77862dfc-6d25-432f-869e-98cc8fa1335f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:28.223229 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-77862dfc-6d25-432f-869e-98cc8fa1335f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:28.225005 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1163/2327] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:08:28 2022] DELETE /identity/v3/projects/e7f5cfc982414991a24dadaea55c21bc => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:28.228849 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f85964-b63a-47e4-88ad-6f5b3237ae5e service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:28.238571 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f85964-b63a-47e4-88ad-6f5b3237ae5e service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:28.295925 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e2dec4-54fb-4902-95f1-8c287c7052d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:28.296591 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2dec4-54fb-4902-95f1-8c287c7052d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:28.296872 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2dec4-54fb-4902-95f1-8c287c7052d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:28.297141 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2dec4-54fb-4902-95f1-8c287c7052d9 admin admin] PATH_INFO: `/v3/users/a9191e3204314c61b60298cbad66218c` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:28.298234 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e2dec4-54fb-4902-95f1-8c287c7052d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9191e3204314c61b60298cbad66218c)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:28.300150 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e2dec4-54fb-4902-95f1-8c287c7052d9 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:28.528512 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-19e2dec4-54fb-4902-95f1-8c287c7052d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9191e3204314c61b60298cbad66218c'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:28.529157 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-19e2dec4-54fb-4902-95f1-8c287c7052d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9191e3204314c61b60298cbad66218c'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:28.612119 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-19e2dec4-54fb-4902-95f1-8c287c7052d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9191e3204314c61b60298cbad66218c'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:28.612776 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-19e2dec4-54fb-4902-95f1-8c287c7052d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9191e3204314c61b60298cbad66218c'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:28.626325 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-19e2dec4-54fb-4902-95f1-8c287c7052d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9191e3204314c61b60298cbad66218c'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:28.639687 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1165/2328] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:08:28 2022] DELETE /identity/v3/users/a9191e3204314c61b60298cbad66218c => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:29.074084 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77862dfc-6d25-432f-869e-98cc8fa1335f admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:29.075865 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-77862dfc-6d25-432f-869e-98cc8fa1335f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:29.103390 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77862dfc-6d25-432f-869e-98cc8fa1335f admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:29.168787 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:29.174520 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e436b253-076a-48d2-9b8f-7ffa685f77d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:29.175164 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436b253-076a-48d2-9b8f-7ffa685f77d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:29.175470 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436b253-076a-48d2-9b8f-7ffa685f77d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:29.175746 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436b253-076a-48d2-9b8f-7ffa685f77d7 admin admin] PATH_INFO: `/v3/projects/e1be28e0feae49f0aaf767af17058024` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:29.177452 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e436b253-076a-48d2-9b8f-7ffa685f77d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1be28e0feae49f0aaf767af17058024)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:29.179400 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e436b253-076a-48d2-9b8f-7ffa685f77d7 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:29.283845 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e436b253-076a-48d2-9b8f-7ffa685f77d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1be28e0feae49f0aaf767af17058024', 'request_id': 'req-e436b253-076a-48d2-9b8f-7ffa685f77d7'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:29.367582 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e436b253-076a-48d2-9b8f-7ffa685f77d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1be28e0feae49f0aaf767af17058024', 'request_id': 'req-e436b253-076a-48d2-9b8f-7ffa685f77d7'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:29.371037 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e436b253-076a-48d2-9b8f-7ffa685f77d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1be28e0feae49f0aaf767af17058024', 'request_id': 'req-e436b253-076a-48d2-9b8f-7ffa685f77d7'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:29.382712 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e436b253-076a-48d2-9b8f-7ffa685f77d7 admin admin] The token cache is being invalidate because project e1be28e0feae49f0aaf767af17058024 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:08:29.383004 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e436b253-076a-48d2-9b8f-7ffa685f77d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:29.383638 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e436b253-076a-48d2-9b8f-7ffa685f77d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:29.385180 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1164/2329] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:08:29 2022] DELETE /identity/v3/projects/e1be28e0feae49f0aaf767af17058024 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:29.390225 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7494ac2-00b5-4bbd-8b89-2b0e9999c5e4 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:29.390853 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7494ac2-00b5-4bbd-8b89-2b0e9999c5e4 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:29.390853 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7494ac2-00b5-4bbd-8b89-2b0e9999c5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:29.412342 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-a7494ac2-00b5-4bbd-8b89-2b0e9999c5e4 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:29.413690 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a7494ac2-00b5-4bbd-8b89-2b0e9999c5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:29.465841 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:29.475045 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1166/2330] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:29.483026 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11523d-3a74-4c26-8295-ea4389227f3d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:29.483322 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11523d-3a74-4c26-8295-ea4389227f3d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:29.483636 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11523d-3a74-4c26-8295-ea4389227f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:29.503771 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-3a11523d-3a74-4c26-8295-ea4389227f3d None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:29.505398 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3a11523d-3a74-4c26-8295-ea4389227f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:29.513965 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1165/2331] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:29.518309 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7494ac2-00b5-4bbd-8b89-2b0e9999c5e4 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:29.528606 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7494ac2-00b5-4bbd-8b89-2b0e9999c5e4 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:29.533151 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d09f8-ee42-4edd-9728-034291490474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:29.533745 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d09f8-ee42-4edd-9728-034291490474 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:29.534023 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d09f8-ee42-4edd-9728-034291490474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:29.534293 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d09f8-ee42-4edd-9728-034291490474 admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:29.534935 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d09f8-ee42-4edd-9728-034291490474 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:08:29.535232 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d09f8-ee42-4edd-9728-034291490474 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:29.537566 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d09f8-ee42-4edd-9728-034291490474 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:29.549815 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1167/2332] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 01:08:29.554201 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a11523d-3a74-4c26-8295-ea4389227f3d None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:29.563265 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a11523d-3a74-4c26-8295-ea4389227f3d None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:29.568338 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f2bf2e-6118-4f95-808d-6ec10e33a9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:29.568976 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2bf2e-6118-4f95-808d-6ec10e33a9d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:29.569281 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2bf2e-6118-4f95-808d-6ec10e33a9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:29.569552 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2bf2e-6118-4f95-808d-6ec10e33a9d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:29.570186 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f2bf2e-6118-4f95-808d-6ec10e33a9d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:29.572215 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f2bf2e-6118-4f95-808d-6ec10e33a9d1 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:29.712217 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1166/2333] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:08:29 2022] POST /identity/v3/projects => generated 379 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:08:29.717595 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d09f8-ee42-4edd-9728-034291490474 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:29.728940 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d09f8-ee42-4edd-9728-034291490474 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:29.789573 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a8b9f-d7d3-4b7b-8640-63417903af9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:29.790189 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a8b9f-d7d3-4b7b-8640-63417903af9d admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:29.790454 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a8b9f-d7d3-4b7b-8640-63417903af9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:29.790727 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a8b9f-d7d3-4b7b-8640-63417903af9d admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:29.791481 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a8b9f-d7d3-4b7b-8640-63417903af9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:29.793447 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a8b9f-d7d3-4b7b-8640-63417903af9d admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:29.906739 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1168/2334] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:08:29 2022] POST /identity/v3/users => generated 290 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:08:29.912308 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f2bf2e-6118-4f95-808d-6ec10e33a9d1 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:29.922617 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f2bf2e-6118-4f95-808d-6ec10e33a9d1 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:29.927249 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8af16-6107-4075-9659-dbd30bcaeb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:29.927840 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8af16-6107-4075-9659-dbd30bcaeb89 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:29.928118 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8af16-6107-4075-9659-dbd30bcaeb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:29.928389 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8af16-6107-4075-9659-dbd30bcaeb89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:29.929051 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b8af16-6107-4075-9659-dbd30bcaeb89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:29.930692 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b8af16-6107-4075-9659-dbd30bcaeb89 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:29.940668 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1167/2335] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:08:29.950439 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a8b9f-d7d3-4b7b-8640-63417903af9d admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:29.961383 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a8b9f-d7d3-4b7b-8640-63417903af9d admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:29.966150 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47daf722-f90a-45e0-b1fe-8ac1490c06aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:29.966747 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47daf722-f90a-45e0-b1fe-8ac1490c06aa admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:29.967028 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47daf722-f90a-45e0-b1fe-8ac1490c06aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:29.967300 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47daf722-f90a-45e0-b1fe-8ac1490c06aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:29.967936 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47daf722-f90a-45e0-b1fe-8ac1490c06aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:29.969563 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47daf722-f90a-45e0-b1fe-8ac1490c06aa admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:29.980814 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1169/2336] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:08:29.984996 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8af16-6107-4075-9659-dbd30bcaeb89 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:29.995564 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8af16-6107-4075-9659-dbd30bcaeb89 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:30.000022 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e903c-119d-4d55-97f5-dacc6c333222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:30.000688 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e903c-119d-4d55-97f5-dacc6c333222 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:30.000965 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e903c-119d-4d55-97f5-dacc6c333222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:30.001234 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e903c-119d-4d55-97f5-dacc6c333222 admin admin] PATH_INFO: `/v3/projects/2a7a101ae4cc47e4accfa240ab3cc7c7/users/5decd7fa630840c7af87e07ec264ff9c/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:30.016604 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e903c-119d-4d55-97f5-dacc6c333222 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a7a101ae4cc47e4accfa240ab3cc7c7, user_id=5decd7fa630840c7af87e07ec264ff9c, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:30.019238 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e903c-119d-4d55-97f5-dacc6c333222 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:30.135831 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1168/2337] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:08:29 2022] PUT /identity/v3/projects/2a7a101ae4cc47e4accfa240ab3cc7c7/users/5decd7fa630840c7af87e07ec264ff9c/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:30.146702 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215fda4-35de-4d6f-891b-826cb3867862 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:30.146994 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215fda4-35de-4d6f-891b-826cb3867862 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:30.147278 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215fda4-35de-4d6f-891b-826cb3867862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:30.182981 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-d215fda4-35de-4d6f-891b-826cb3867862 None None] MFA Rules not processed for user `5decd7fa630840c7af87e07ec264ff9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:30.184657 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-d215fda4-35de-4d6f-891b-826cb3867862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:30.270514 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1170/2338] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:30.275353 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e903c-119d-4d55-97f5-dacc6c333222 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:30.340581 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e903c-119d-4d55-97f5-dacc6c333222 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:30.345332 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c164479d-7a56-4d1e-9129-76c25c5984f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:30.347704 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164479d-7a56-4d1e-9129-76c25c5984f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:30.347704 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164479d-7a56-4d1e-9129-76c25c5984f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:30.347704 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164479d-7a56-4d1e-9129-76c25c5984f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:30.347704 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c164479d-7a56-4d1e-9129-76c25c5984f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:30.349574 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c164479d-7a56-4d1e-9129-76c25c5984f5 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:30.450930 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1169/2339] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:08:30 2022] POST /identity/v3/projects => generated 379 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:08:30.455156 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215fda4-35de-4d6f-891b-826cb3867862 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:30.466529 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215fda4-35de-4d6f-891b-826cb3867862 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:30.517816 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:30.525110 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab75d61e-534a-4922-8505-dc72617363ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:30.525719 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75d61e-534a-4922-8505-dc72617363ad admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:30.526000 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75d61e-534a-4922-8505-dc72617363ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:30.526274 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75d61e-534a-4922-8505-dc72617363ad admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:30.526896 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab75d61e-534a-4922-8505-dc72617363ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:30.528869 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab75d61e-534a-4922-8505-dc72617363ad admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:30.647435 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1171/2340] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:08:30 2022] POST /identity/v3/users => generated 296 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:08:30.651471 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c164479d-7a56-4d1e-9129-76c25c5984f5 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:30.662618 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c164479d-7a56-4d1e-9129-76c25c5984f5 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:30.667471 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473899ef-ee69-4b06-a39b-9fc6da0f5507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:30.668075 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473899ef-ee69-4b06-a39b-9fc6da0f5507 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:30.668354 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473899ef-ee69-4b06-a39b-9fc6da0f5507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:30.668667 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473899ef-ee69-4b06-a39b-9fc6da0f5507 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:30.669305 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473899ef-ee69-4b06-a39b-9fc6da0f5507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:30.670946 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473899ef-ee69-4b06-a39b-9fc6da0f5507 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:30.673350 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:30.683034 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1170/2341] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 01:08:30.687284 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab75d61e-534a-4922-8505-dc72617363ad admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:30.697787 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab75d61e-534a-4922-8505-dc72617363ad admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:30.702619 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a89c93-b94b-47b8-9914-19f717ed5140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:30.703190 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a89c93-b94b-47b8-9914-19f717ed5140 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:30.703468 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a89c93-b94b-47b8-9914-19f717ed5140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:30.703740 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a89c93-b94b-47b8-9914-19f717ed5140 admin admin] PATH_INFO: `/v3/projects/e217354fdebf4e2f9fe2ea4553c4f61b/users/388c1e701c474c20a29bd456dea1a79d/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:30.718942 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a89c93-b94b-47b8-9914-19f717ed5140 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e217354fdebf4e2f9fe2ea4553c4f61b, user_id=388c1e701c474c20a29bd456dea1a79d, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:30.721611 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a89c93-b94b-47b8-9914-19f717ed5140 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:30.813242 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1172/2342] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:08:30 2022] PUT /identity/v3/projects/e217354fdebf4e2f9fe2ea4553c4f61b/users/388c1e701c474c20a29bd456dea1a79d/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:30.817518 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473899ef-ee69-4b06-a39b-9fc6da0f5507 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:30.828059 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473899ef-ee69-4b06-a39b-9fc6da0f5507 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:30.882569 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840c9a8-bef9-4548-b67f-f6954c4dd192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:30.883175 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840c9a8-bef9-4548-b67f-f6954c4dd192 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:30.883451 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840c9a8-bef9-4548-b67f-f6954c4dd192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:30.883726 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840c9a8-bef9-4548-b67f-f6954c4dd192 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:30.884410 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b840c9a8-bef9-4548-b67f-f6954c4dd192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:30.886007 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b840c9a8-bef9-4548-b67f-f6954c4dd192 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:30.896413 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1171/2343] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:08:30.900802 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a89c93-b94b-47b8-9914-19f717ed5140 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:30.911797 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a89c93-b94b-47b8-9914-19f717ed5140 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:30.917035 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b71487-64a5-4cda-b036-cf7fe14c9335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:30.917722 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b71487-64a5-4cda-b036-cf7fe14c9335 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:30.918121 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b71487-64a5-4cda-b036-cf7fe14c9335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:30.918409 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b71487-64a5-4cda-b036-cf7fe14c9335 admin admin] PATH_INFO: `/v3/projects/e217354fdebf4e2f9fe2ea4553c4f61b/users/388c1e701c474c20a29bd456dea1a79d/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:30.920564 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b71487-64a5-4cda-b036-cf7fe14c9335 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e217354fdebf4e2f9fe2ea4553c4f61b, user_id=388c1e701c474c20a29bd456dea1a79d, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:30.923245 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b71487-64a5-4cda-b036-cf7fe14c9335 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:30.934556 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1173/2344] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:08:30 2022] PUT /identity/v3/projects/e217354fdebf4e2f9fe2ea4553c4f61b/users/388c1e701c474c20a29bd456dea1a79d/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:30.946189 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795e537-ecf2-4348-a192-a428c327b296 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:30.946514 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795e537-ecf2-4348-a192-a428c327b296 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:30.946790 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795e537-ecf2-4348-a192-a428c327b296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:30.975642 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-b795e537-ecf2-4348-a192-a428c327b296 None None] MFA Rules not processed for user `388c1e701c474c20a29bd456dea1a79d`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:30.977033 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b795e537-ecf2-4348-a192-a428c327b296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:31.048221 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1172/2345] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:30 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:31.061482 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b71487-64a5-4cda-b036-cf7fe14c9335 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:31.063169 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b1b71487-64a5-4cda-b036-cf7fe14c9335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:31.122985 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b71487-64a5-4cda-b036-cf7fe14c9335 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:31.126857 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d39cf-8778-46d8-a71a-800539faf518 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:31.127457 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d39cf-8778-46d8-a71a-800539faf518 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:31.127734 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d39cf-8778-46d8-a71a-800539faf518 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:31.128004 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d39cf-8778-46d8-a71a-800539faf518 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:31.138381 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d39cf-8778-46d8-a71a-800539faf518 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:31.139838 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d39cf-8778-46d8-a71a-800539faf518 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:31.149825 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1174/2346] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:08:31 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:08:32.411205 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795e537-ecf2-4348-a192-a428c327b296 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:32.420793 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795e537-ecf2-4348-a192-a428c327b296 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:32.425551 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9132c-4ba1-4972-afa4-475856ba9673 tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '388c1e701c474c20a29bd456dea1a79d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e217354fdebf4e2f9fe2ea4553c4f61b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:32.426147 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9132c-4ba1-4972-afa4-475856ba9673 tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:32.426433 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9132c-4ba1-4972-afa4-475856ba9673 tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:32.426706 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9132c-4ba1-4972-afa4-475856ba9673 tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:32.427838 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9132c-4ba1-4972-afa4-475856ba9673 tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:08:32.428179 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9132c-4ba1-4972-afa4-475856ba9673 tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:32.429935 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9132c-4ba1-4972-afa4-475856ba9673 tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:32.441533 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1173/2347] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:08:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:08:32.446425 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d39cf-8778-46d8-a71a-800539faf518 service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:32.455950 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d39cf-8778-46d8-a71a-800539faf518 service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:32.460913 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9852a-11e2-4787-b505-73ea5fecf013 tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '388c1e701c474c20a29bd456dea1a79d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e217354fdebf4e2f9fe2ea4553c4f61b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:32.461623 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9852a-11e2-4787-b505-73ea5fecf013 tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:32.461910 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9852a-11e2-4787-b505-73ea5fecf013 tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:32.462232 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9852a-11e2-4787-b505-73ea5fecf013 tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:32.462814 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b9852a-11e2-4787-b505-73ea5fecf013 tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:32.464907 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b9852a-11e2-4787-b505-73ea5fecf013 tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:32.568943 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1175/2348] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:08:32 2022] POST /identity/v3/projects => generated 354 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:08:32.570450 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9132c-4ba1-4972-afa4-475856ba9673 tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:32.580776 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9132c-4ba1-4972-afa4-475856ba9673 tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:32.639207 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39523fbd-b14b-4ed5-ae80-b1283a10a871 tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '388c1e701c474c20a29bd456dea1a79d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e217354fdebf4e2f9fe2ea4553c4f61b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:32.639839 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39523fbd-b14b-4ed5-ae80-b1283a10a871 tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:32.640120 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39523fbd-b14b-4ed5-ae80-b1283a10a871 tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:32.640401 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39523fbd-b14b-4ed5-ae80-b1283a10a871 tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:32.641154 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39523fbd-b14b-4ed5-ae80-b1283a10a871 tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:08:32.648858 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39523fbd-b14b-4ed5-ae80-b1283a10a871 tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:32.648858 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39523fbd-b14b-4ed5-ae80-b1283a10a871 tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:32.661021 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1174/2349] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:08:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:08:32.662433 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:32.699144 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9852a-11e2-4787-b505-73ea5fecf013 tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:32.712752 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9852a-11e2-4787-b505-73ea5fecf013 tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:32.715585 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b2771-c6a4-4b45-9d1f-d2932a660aae tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '388c1e701c474c20a29bd456dea1a79d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e217354fdebf4e2f9fe2ea4553c4f61b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:32.716313 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b2771-c6a4-4b45-9d1f-d2932a660aae tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:32.716742 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b2771-c6a4-4b45-9d1f-d2932a660aae tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:32.717115 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b2771-c6a4-4b45-9d1f-d2932a660aae tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] PATH_INFO: `/v3/projects/01ed37e5b75a467b9d97fe97efa63149` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:32.718364 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3b2771-c6a4-4b45-9d1f-d2932a660aae tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] RBAC: Authorizing `identity:delete_project(project_id=01ed37e5b75a467b9d97fe97efa63149)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:32.720737 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3b2771-c6a4-4b45-9d1f-d2932a660aae tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:32.832208 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2c3b2771-c6a4-4b45-9d1f-d2932a660aae tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01ed37e5b75a467b9d97fe97efa63149', 'request_id': 'req-2c3b2771-c6a4-4b45-9d1f-d2932a660aae'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:32.923096 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2c3b2771-c6a4-4b45-9d1f-d2932a660aae tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01ed37e5b75a467b9d97fe97efa63149', 'request_id': 'req-2c3b2771-c6a4-4b45-9d1f-d2932a660aae'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:32.929492 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2c3b2771-c6a4-4b45-9d1f-d2932a660aae tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01ed37e5b75a467b9d97fe97efa63149', 'request_id': 'req-2c3b2771-c6a4-4b45-9d1f-d2932a660aae'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:32.941547 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2c3b2771-c6a4-4b45-9d1f-d2932a660aae tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] The token cache is being invalidate because project 01ed37e5b75a467b9d97fe97efa63149 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:08:32.941833 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2c3b2771-c6a4-4b45-9d1f-d2932a660aae tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:32.942478 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2c3b2771-c6a4-4b45-9d1f-d2932a660aae tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:32.944597 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1176/2350] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:08:32 2022] DELETE /identity/v3/projects/01ed37e5b75a467b9d97fe97efa63149 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:33.647301 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39523fbd-b14b-4ed5-ae80-b1283a10a871 tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:33.649160 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-39523fbd-b14b-4ed5-ae80-b1283a10a871 tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:33.718315 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39523fbd-b14b-4ed5-ae80-b1283a10a871 tempest-VolumeQuotaClassesTest-406999367 tempest-VolumeQuotaClassesTest-406999367-project-admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:33.778157 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171de6b7-7e55-4de3-b630-ee722f2e49c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:33.783349 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171de6b7-7e55-4de3-b630-ee722f2e49c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:33.783349 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171de6b7-7e55-4de3-b630-ee722f2e49c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:33.783349 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171de6b7-7e55-4de3-b630-ee722f2e49c4 admin admin] PATH_INFO: `/v3/users/5decd7fa630840c7af87e07ec264ff9c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:33.783349 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171de6b7-7e55-4de3-b630-ee722f2e49c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5decd7fa630840c7af87e07ec264ff9c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:33.783349 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171de6b7-7e55-4de3-b630-ee722f2e49c4 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:33.999824 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-171de6b7-7e55-4de3-b630-ee722f2e49c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5decd7fa630840c7af87e07ec264ff9c'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:34.000621 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-171de6b7-7e55-4de3-b630-ee722f2e49c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5decd7fa630840c7af87e07ec264ff9c'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:34.008195 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-171de6b7-7e55-4de3-b630-ee722f2e49c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5decd7fa630840c7af87e07ec264ff9c'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:34.082971 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-171de6b7-7e55-4de3-b630-ee722f2e49c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5decd7fa630840c7af87e07ec264ff9c'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:34.083792 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-171de6b7-7e55-4de3-b630-ee722f2e49c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5decd7fa630840c7af87e07ec264ff9c'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:34.104391 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1175/2351] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:08:33 2022] DELETE /identity/v3/users/5decd7fa630840c7af87e07ec264ff9c => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:34.108912 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b10918-4e01-4730-92ee-32b067d5d1a6 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:34.109398 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b10918-4e01-4730-92ee-32b067d5d1a6 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:34.109505 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b10918-4e01-4730-92ee-32b067d5d1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:34.128634 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-56b10918-4e01-4730-92ee-32b067d5d1a6 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:34.129830 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-56b10918-4e01-4730-92ee-32b067d5d1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:34.135223 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:34.193632 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1177/2352] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:34.210623 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171de6b7-7e55-4de3-b630-ee722f2e49c4 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:34.212176 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-171de6b7-7e55-4de3-b630-ee722f2e49c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:34.251250 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:34.278913 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171de6b7-7e55-4de3-b630-ee722f2e49c4 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:34.282801 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2e860f-2df2-4401-b1db-3a9c0c148bf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:34.283520 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e860f-2df2-4401-b1db-3a9c0c148bf5 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:34.283808 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e860f-2df2-4401-b1db-3a9c0c148bf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:34.284084 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e860f-2df2-4401-b1db-3a9c0c148bf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:34.294141 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2e860f-2df2-4401-b1db-3a9c0c148bf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:34.295655 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2e860f-2df2-4401-b1db-3a9c0c148bf5 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:34.305863 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1176/2353] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:08:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:08:34.768237 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b10918-4e01-4730-92ee-32b067d5d1a6 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:34.769849 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-56b10918-4e01-4730-92ee-32b067d5d1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:34.783315 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b10918-4e01-4730-92ee-32b067d5d1a6 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:34.793235 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac3919-2d1e-4457-9740-41dca929a564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:34.793235 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac3919-2d1e-4457-9740-41dca929a564 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:34.793235 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac3919-2d1e-4457-9740-41dca929a564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:34.793235 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac3919-2d1e-4457-9740-41dca929a564 admin admin] PATH_INFO: `/v3/projects/2a7a101ae4cc47e4accfa240ab3cc7c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:34.793852 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac3919-2d1e-4457-9740-41dca929a564 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a7a101ae4cc47e4accfa240ab3cc7c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:34.795985 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac3919-2d1e-4457-9740-41dca929a564 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:34.901115 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b4ac3919-2d1e-4457-9740-41dca929a564 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a7a101ae4cc47e4accfa240ab3cc7c7', 'request_id': 'req-b4ac3919-2d1e-4457-9740-41dca929a564'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:34.995332 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b4ac3919-2d1e-4457-9740-41dca929a564 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a7a101ae4cc47e4accfa240ab3cc7c7', 'request_id': 'req-b4ac3919-2d1e-4457-9740-41dca929a564'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:35.001735 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b4ac3919-2d1e-4457-9740-41dca929a564 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a7a101ae4cc47e4accfa240ab3cc7c7', 'request_id': 'req-b4ac3919-2d1e-4457-9740-41dca929a564'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:35.014073 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b4ac3919-2d1e-4457-9740-41dca929a564 admin admin] The token cache is being invalidate because project 2a7a101ae4cc47e4accfa240ab3cc7c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:08:35.014366 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b4ac3919-2d1e-4457-9740-41dca929a564 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:35.015158 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b4ac3919-2d1e-4457-9740-41dca929a564 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:35.016684 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1178/2354] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:08:34 2022] DELETE /identity/v3/projects/2a7a101ae4cc47e4accfa240ab3cc7c7 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:35.020782 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2e860f-2df2-4401-b1db-3a9c0c148bf5 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:35.030842 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2e860f-2df2-4401-b1db-3a9c0c148bf5 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:35.091512 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be051ee1-2d91-43c1-8424-1ab472071ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:35.092182 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be051ee1-2d91-43c1-8424-1ab472071ea5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:35.092489 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be051ee1-2d91-43c1-8424-1ab472071ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:35.099587 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be051ee1-2d91-43c1-8424-1ab472071ea5 admin admin] PATH_INFO: `/v3/users/388c1e701c474c20a29bd456dea1a79d` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:35.099587 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be051ee1-2d91-43c1-8424-1ab472071ea5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=388c1e701c474c20a29bd456dea1a79d)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:35.099587 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be051ee1-2d91-43c1-8424-1ab472071ea5 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:35.285870 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-be051ee1-2d91-43c1-8424-1ab472071ea5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '388c1e701c474c20a29bd456dea1a79d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:35.287127 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-be051ee1-2d91-43c1-8424-1ab472071ea5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '388c1e701c474c20a29bd456dea1a79d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:35.294705 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-be051ee1-2d91-43c1-8424-1ab472071ea5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '388c1e701c474c20a29bd456dea1a79d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:35.369084 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-be051ee1-2d91-43c1-8424-1ab472071ea5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '388c1e701c474c20a29bd456dea1a79d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:35.369665 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-be051ee1-2d91-43c1-8424-1ab472071ea5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '388c1e701c474c20a29bd456dea1a79d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:35.393798 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1177/2355] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:08:35 2022] DELETE /identity/v3/users/388c1e701c474c20a29bd456dea1a79d => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:35.852938 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac3919-2d1e-4457-9740-41dca929a564 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:35.854415 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b4ac3919-2d1e-4457-9740-41dca929a564 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:35.878301 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac3919-2d1e-4457-9740-41dca929a564 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:35.940478 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74eb18b-7359-418c-8390-ea2246f1ea5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:35.941180 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74eb18b-7359-418c-8390-ea2246f1ea5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:35.941492 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74eb18b-7359-418c-8390-ea2246f1ea5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:35.941770 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74eb18b-7359-418c-8390-ea2246f1ea5f admin admin] PATH_INFO: `/v3/projects/e217354fdebf4e2f9fe2ea4553c4f61b` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:35.943646 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74eb18b-7359-418c-8390-ea2246f1ea5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e217354fdebf4e2f9fe2ea4553c4f61b)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:35.945592 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74eb18b-7359-418c-8390-ea2246f1ea5f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:36.029756 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:36.039965 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f74eb18b-7359-418c-8390-ea2246f1ea5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e217354fdebf4e2f9fe2ea4553c4f61b', 'request_id': 'req-f74eb18b-7359-418c-8390-ea2246f1ea5f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:36.111578 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f74eb18b-7359-418c-8390-ea2246f1ea5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e217354fdebf4e2f9fe2ea4553c4f61b', 'request_id': 'req-f74eb18b-7359-418c-8390-ea2246f1ea5f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:36.118292 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f74eb18b-7359-418c-8390-ea2246f1ea5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e217354fdebf4e2f9fe2ea4553c4f61b', 'request_id': 'req-f74eb18b-7359-418c-8390-ea2246f1ea5f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:36.129462 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f74eb18b-7359-418c-8390-ea2246f1ea5f admin admin] The token cache is being invalidate because project e217354fdebf4e2f9fe2ea4553c4f61b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:08:36.129801 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f74eb18b-7359-418c-8390-ea2246f1ea5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:36.131436 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f74eb18b-7359-418c-8390-ea2246f1ea5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:36.133073 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1179/2356] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:08:35 2022] DELETE /identity/v3/projects/e217354fdebf4e2f9fe2ea4553c4f61b => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:36.152966 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd6091-124a-447e-9a4a-d43879300c18 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:36.152966 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd6091-124a-447e-9a4a-d43879300c18 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:36.152966 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd6091-124a-447e-9a4a-d43879300c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:36.165594 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-34cd6091-124a-447e-9a4a-d43879300c18 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:36.166884 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-34cd6091-124a-447e-9a4a-d43879300c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:36.236717 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:36.253211 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1178/2357] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:08:36.261799 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a92e4-8472-4aac-b0d6-80b42c915172 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:36.262086 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a92e4-8472-4aac-b0d6-80b42c915172 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:36.262358 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a92e4-8472-4aac-b0d6-80b42c915172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:36.298525 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-2e0a92e4-8472-4aac-b0d6-80b42c915172 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:36.306633 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2e0a92e4-8472-4aac-b0d6-80b42c915172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:36.320343 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1180/2358] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:36.330644 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd6091-124a-447e-9a4a-d43879300c18 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:36.344950 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd6091-124a-447e-9a4a-d43879300c18 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:36.356885 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc226190-953b-4b78-b739-74948a3cc725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:36.357372 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc226190-953b-4b78-b739-74948a3cc725 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:36.357656 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc226190-953b-4b78-b739-74948a3cc725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:36.357934 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc226190-953b-4b78-b739-74948a3cc725 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:36.358602 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc226190-953b-4b78-b739-74948a3cc725 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:08:36.358954 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc226190-953b-4b78-b739-74948a3cc725 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:36.360583 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc226190-953b-4b78-b739-74948a3cc725 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:36.386845 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1179/2359] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:08:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:08:36.391438 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a92e4-8472-4aac-b0d6-80b42c915172 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:36.405998 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a92e4-8472-4aac-b0d6-80b42c915172 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:36.429306 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95342658-166a-4cec-9152-90704641a2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:36.430199 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95342658-166a-4cec-9152-90704641a2f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:36.430773 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95342658-166a-4cec-9152-90704641a2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:36.431201 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95342658-166a-4cec-9152-90704641a2f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:36.432042 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95342658-166a-4cec-9152-90704641a2f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:36.434746 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95342658-166a-4cec-9152-90704641a2f8 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:36.540433 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1181/2360] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:08:36 2022] POST /identity/v3/projects => generated 371 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:08:36.545883 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc226190-953b-4b78-b739-74948a3cc725 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:36.557026 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc226190-953b-4b78-b739-74948a3cc725 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:36.618926 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe00b5-ef70-4d26-b573-7f8638cf2b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:36.621211 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe00b5-ef70-4d26-b573-7f8638cf2b56 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:36.621552 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe00b5-ef70-4d26-b573-7f8638cf2b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:36.621799 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe00b5-ef70-4d26-b573-7f8638cf2b56 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:36.622495 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe00b5-ef70-4d26-b573-7f8638cf2b56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:36.624620 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe00b5-ef70-4d26-b573-7f8638cf2b56 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:36.728659 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1180/2361] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:08:36 2022] POST /identity/v3/users => generated 286 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:08:36.732488 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95342658-166a-4cec-9152-90704641a2f8 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:36.751712 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95342658-166a-4cec-9152-90704641a2f8 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:36.759155 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0061792d-9b45-45be-a3dd-9719c085ecff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:36.760005 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0061792d-9b45-45be-a3dd-9719c085ecff admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:36.760371 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0061792d-9b45-45be-a3dd-9719c085ecff admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:36.760768 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0061792d-9b45-45be-a3dd-9719c085ecff admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:36.761964 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0061792d-9b45-45be-a3dd-9719c085ecff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:36.763973 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0061792d-9b45-45be-a3dd-9719c085ecff admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:36.775771 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1182/2362] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 01:08:36.780213 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe00b5-ef70-4d26-b573-7f8638cf2b56 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:36.792147 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe00b5-ef70-4d26-b573-7f8638cf2b56 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:36.797230 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ee17a1-a61a-414d-9295-24472ea42d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:36.800691 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee17a1-a61a-414d-9295-24472ea42d06 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:36.801101 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee17a1-a61a-414d-9295-24472ea42d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:36.801510 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee17a1-a61a-414d-9295-24472ea42d06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:36.803658 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ee17a1-a61a-414d-9295-24472ea42d06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:36.805790 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ee17a1-a61a-414d-9295-24472ea42d06 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:36.829151 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1181/2363] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:08:36.834016 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0061792d-9b45-45be-a3dd-9719c085ecff admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:36.846067 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0061792d-9b45-45be-a3dd-9719c085ecff admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:36.853027 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f26cc8f-8bde-4a3f-b6e3-545712fcba99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:36.853854 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26cc8f-8bde-4a3f-b6e3-545712fcba99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:36.854321 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26cc8f-8bde-4a3f-b6e3-545712fcba99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:36.854603 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26cc8f-8bde-4a3f-b6e3-545712fcba99 admin admin] PATH_INFO: `/v3/projects/8482fcb7639e47f68e8718a2e4499fe2/users/0ca29c18830a4cb5b13fb3c291032493/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:36.873617 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f26cc8f-8bde-4a3f-b6e3-545712fcba99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8482fcb7639e47f68e8718a2e4499fe2, user_id=0ca29c18830a4cb5b13fb3c291032493, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:36.876615 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f26cc8f-8bde-4a3f-b6e3-545712fcba99 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:36.965155 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1183/2364] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:08:36 2022] PUT /identity/v3/projects/8482fcb7639e47f68e8718a2e4499fe2/users/0ca29c18830a4cb5b13fb3c291032493/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:36.977759 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa8ecd-67ad-4e98-a9ca-31f874ef662a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:36.978048 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa8ecd-67ad-4e98-a9ca-31f874ef662a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:36.978328 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa8ecd-67ad-4e98-a9ca-31f874ef662a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:37.028692 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-fffa8ecd-67ad-4e98-a9ca-31f874ef662a None None] MFA Rules not processed for user `0ca29c18830a4cb5b13fb3c291032493`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:37.033091 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-fffa8ecd-67ad-4e98-a9ca-31f874ef662a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:37.119917 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1182/2365] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:08:37.132561 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f26cc8f-8bde-4a3f-b6e3-545712fcba99 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:37.135626 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-4f26cc8f-8bde-4a3f-b6e3-545712fcba99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:37.201867 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f26cc8f-8bde-4a3f-b6e3-545712fcba99 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:37.205703 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00f23f-1afe-4f5a-8e50-788dbcce5350 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:37.206355 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00f23f-1afe-4f5a-8e50-788dbcce5350 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:37.206641 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00f23f-1afe-4f5a-8e50-788dbcce5350 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:37.210105 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00f23f-1afe-4f5a-8e50-788dbcce5350 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:37.215734 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed00f23f-1afe-4f5a-8e50-788dbcce5350 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:37.217243 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed00f23f-1afe-4f5a-8e50-788dbcce5350 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:37.234505 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1184/2366] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:08:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:08:37.257072 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa8ecd-67ad-4e98-a9ca-31f874ef662a None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:37.316003 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:37.333334 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa8ecd-67ad-4e98-a9ca-31f874ef662a None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:37.338368 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6b316-a84c-4b7d-85f7-57456de31567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:37.338997 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6b316-a84c-4b7d-85f7-57456de31567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:37.339275 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6b316-a84c-4b7d-85f7-57456de31567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:37.339575 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6b316-a84c-4b7d-85f7-57456de31567 admin admin] PATH_INFO: `/v3/users/0ca29c18830a4cb5b13fb3c291032493` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:37.340763 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd6b316-a84c-4b7d-85f7-57456de31567 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ca29c18830a4cb5b13fb3c291032493)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:37.342692 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd6b316-a84c-4b7d-85f7-57456de31567 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:37.688479 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-0fd6b316-a84c-4b7d-85f7-57456de31567 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ca29c18830a4cb5b13fb3c291032493'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:37.689153 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-0fd6b316-a84c-4b7d-85f7-57456de31567 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ca29c18830a4cb5b13fb3c291032493'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:37.696109 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-0fd6b316-a84c-4b7d-85f7-57456de31567 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ca29c18830a4cb5b13fb3c291032493'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:37.779995 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-0fd6b316-a84c-4b7d-85f7-57456de31567 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ca29c18830a4cb5b13fb3c291032493'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:37.780664 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-0fd6b316-a84c-4b7d-85f7-57456de31567 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ca29c18830a4cb5b13fb3c291032493'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:37.799470 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1183/2367] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:08:37 2022] DELETE /identity/v3/users/0ca29c18830a4cb5b13fb3c291032493 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:37.804661 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d5565-4c9f-46d1-942d-c3c06df09fb1 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:37.804951 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d5565-4c9f-46d1-942d-c3c06df09fb1 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:37.805227 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d5565-4c9f-46d1-942d-c3c06df09fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:37.825078 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-bc9d5565-4c9f-46d1-942d-c3c06df09fb1 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:37.826274 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-bc9d5565-4c9f-46d1-942d-c3c06df09fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:37.885246 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1185/2368] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:37.895295 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6b316-a84c-4b7d-85f7-57456de31567 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:37.897039 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0fd6b316-a84c-4b7d-85f7-57456de31567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:37.957761 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6b316-a84c-4b7d-85f7-57456de31567 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:37.961547 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a6bfc-8c47-43b7-9e43-7d504fb39c4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:37.962152 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a6bfc-8c47-43b7-9e43-7d504fb39c4f service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:37.962433 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a6bfc-8c47-43b7-9e43-7d504fb39c4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:37.962708 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a6bfc-8c47-43b7-9e43-7d504fb39c4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:37.971684 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38a6bfc-8c47-43b7-9e43-7d504fb39c4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:37.973144 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38a6bfc-8c47-43b7-9e43-7d504fb39c4f service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:37.983008 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1184/2369] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:08:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:08:38.471679 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d5565-4c9f-46d1-942d-c3c06df09fb1 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:38.476693 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-bc9d5565-4c9f-46d1-942d-c3c06df09fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:38.481984 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:38.486249 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d5565-4c9f-46d1-942d-c3c06df09fb1 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:38.491919 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa12e1d-71ff-4a94-8d26-bb64032ff6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:38.492661 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa12e1d-71ff-4a94-8d26-bb64032ff6c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:38.492916 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa12e1d-71ff-4a94-8d26-bb64032ff6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:38.493193 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa12e1d-71ff-4a94-8d26-bb64032ff6c4 admin admin] PATH_INFO: `/v3/projects/8482fcb7639e47f68e8718a2e4499fe2` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:38.494330 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa12e1d-71ff-4a94-8d26-bb64032ff6c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8482fcb7639e47f68e8718a2e4499fe2)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:38.496273 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa12e1d-71ff-4a94-8d26-bb64032ff6c4 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:38.618407 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-caa12e1d-71ff-4a94-8d26-bb64032ff6c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8482fcb7639e47f68e8718a2e4499fe2', 'request_id': 'req-caa12e1d-71ff-4a94-8d26-bb64032ff6c4'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:38.720918 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-caa12e1d-71ff-4a94-8d26-bb64032ff6c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8482fcb7639e47f68e8718a2e4499fe2', 'request_id': 'req-caa12e1d-71ff-4a94-8d26-bb64032ff6c4'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:38.729993 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-caa12e1d-71ff-4a94-8d26-bb64032ff6c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8482fcb7639e47f68e8718a2e4499fe2', 'request_id': 'req-caa12e1d-71ff-4a94-8d26-bb64032ff6c4'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:38.740153 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-caa12e1d-71ff-4a94-8d26-bb64032ff6c4 admin admin] The token cache is being invalidate because project 8482fcb7639e47f68e8718a2e4499fe2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:08:38.740550 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-caa12e1d-71ff-4a94-8d26-bb64032ff6c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:38.741231 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-caa12e1d-71ff-4a94-8d26-bb64032ff6c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:38.742848 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1186/2370] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:08:38 2022] DELETE /identity/v3/projects/8482fcb7639e47f68e8718a2e4499fe2 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:38.747745 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9b22a-a00d-4a00-ac0d-101163875320 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:38.748192 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9b22a-a00d-4a00-ac0d-101163875320 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:38.748444 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9b22a-a00d-4a00-ac0d-101163875320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:38.771347 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1ec9b22a-a00d-4a00-ac0d-101163875320 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:38.771347 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1ec9b22a-a00d-4a00-ac0d-101163875320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:38.845339 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1185/2371] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:38.853324 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda0d57-0b3a-447c-a08e-d2f7b2a2876d None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:38.853616 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda0d57-0b3a-447c-a08e-d2f7b2a2876d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:38.853894 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda0d57-0b3a-447c-a08e-d2f7b2a2876d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:38.877346 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-2bda0d57-0b3a-447c-a08e-d2f7b2a2876d None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:38.878579 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2bda0d57-0b3a-447c-a08e-d2f7b2a2876d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:38.887530 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1187/2372] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:38.891919 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9b22a-a00d-4a00-ac0d-101163875320 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:38.901770 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9b22a-a00d-4a00-ac0d-101163875320 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:38.906332 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c556753c-3968-4eb4-95ab-558498bfbab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:38.906929 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556753c-3968-4eb4-95ab-558498bfbab8 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:38.907212 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556753c-3968-4eb4-95ab-558498bfbab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:38.907501 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556753c-3968-4eb4-95ab-558498bfbab8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:38.908128 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c556753c-3968-4eb4-95ab-558498bfbab8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:08:38.908431 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c556753c-3968-4eb4-95ab-558498bfbab8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:38.910042 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c556753c-3968-4eb4-95ab-558498bfbab8 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:38.912129 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:38.923229 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1186/2373] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 01:08:38.928359 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda0d57-0b3a-447c-a08e-d2f7b2a2876d None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:38.937533 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda0d57-0b3a-447c-a08e-d2f7b2a2876d None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:38.942254 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b97b0d-18f7-4762-8a0c-64c3a53db8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:38.943043 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b97b0d-18f7-4762-8a0c-64c3a53db8da admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:38.943329 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b97b0d-18f7-4762-8a0c-64c3a53db8da admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:38.943602 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b97b0d-18f7-4762-8a0c-64c3a53db8da admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:38.944263 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b97b0d-18f7-4762-8a0c-64c3a53db8da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:38.946329 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b97b0d-18f7-4762-8a0c-64c3a53db8da admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:39.028527 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1188/2374] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:08:38 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:08:39.031398 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c556753c-3968-4eb4-95ab-558498bfbab8 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:39.041771 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c556753c-3968-4eb4-95ab-558498bfbab8 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:39.165999 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9f717c-c917-4a8f-a941-880ca0066ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:39.166830 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f717c-c917-4a8f-a941-880ca0066ecb admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:39.167281 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f717c-c917-4a8f-a941-880ca0066ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:39.167631 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f717c-c917-4a8f-a941-880ca0066ecb admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:39.168337 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9f717c-c917-4a8f-a941-880ca0066ecb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:39.170469 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9f717c-c917-4a8f-a941-880ca0066ecb admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:39.277236 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1187/2375] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:08:39 2022] POST /identity/v3/users => generated 293 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:08:39.281423 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b97b0d-18f7-4762-8a0c-64c3a53db8da admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:39.292874 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b97b0d-18f7-4762-8a0c-64c3a53db8da admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:39.298086 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003719a9-0b58-4dc2-bc2b-e20ac06de583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:39.298692 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003719a9-0b58-4dc2-bc2b-e20ac06de583 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:39.299008 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003719a9-0b58-4dc2-bc2b-e20ac06de583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:39.299283 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003719a9-0b58-4dc2-bc2b-e20ac06de583 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:39.299966 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003719a9-0b58-4dc2-bc2b-e20ac06de583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:39.301577 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003719a9-0b58-4dc2-bc2b-e20ac06de583 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:39.312671 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1189/2376] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 01:08:39.316895 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9f717c-c917-4a8f-a941-880ca0066ecb admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:39.327471 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9f717c-c917-4a8f-a941-880ca0066ecb admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:39.331978 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c6a71-e64e-4183-9aad-306d7ce50fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:39.332609 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c6a71-e64e-4183-9aad-306d7ce50fed admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:39.332882 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c6a71-e64e-4183-9aad-306d7ce50fed admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:39.333178 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c6a71-e64e-4183-9aad-306d7ce50fed admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:39.333805 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6c6a71-e64e-4183-9aad-306d7ce50fed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:39.335406 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6c6a71-e64e-4183-9aad-306d7ce50fed admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:39.345176 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1188/2377] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 01:08:39.349857 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003719a9-0b58-4dc2-bc2b-e20ac06de583 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:39.359919 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003719a9-0b58-4dc2-bc2b-e20ac06de583 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:39.365271 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3210d1f-e314-49c4-b2f0-660612a11d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:39.365943 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3210d1f-e314-49c4-b2f0-660612a11d7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:39.366220 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3210d1f-e314-49c4-b2f0-660612a11d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:39.366494 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3210d1f-e314-49c4-b2f0-660612a11d7b admin admin] PATH_INFO: `/v3/projects/1678777938d84d40a794fa79aafc5e42/users/2f212544d92e4380972f25fa83598d23/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:39.381000 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3210d1f-e314-49c4-b2f0-660612a11d7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1678777938d84d40a794fa79aafc5e42, user_id=2f212544d92e4380972f25fa83598d23, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:39.383689 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3210d1f-e314-49c4-b2f0-660612a11d7b admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:39.460435 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1190/2378] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:08:39 2022] PUT /identity/v3/projects/1678777938d84d40a794fa79aafc5e42/users/2f212544d92e4380972f25fa83598d23/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:39.470741 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361eb40a-1d48-415d-81c8-ad1581b2c3f3 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:39.471023 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361eb40a-1d48-415d-81c8-ad1581b2c3f3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:39.471300 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361eb40a-1d48-415d-81c8-ad1581b2c3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:39.499096 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-361eb40a-1d48-415d-81c8-ad1581b2c3f3 None None] MFA Rules not processed for user `2f212544d92e4380972f25fa83598d23`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:39.500339 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-361eb40a-1d48-415d-81c8-ad1581b2c3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:39.567344 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1189/2379] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:39.572035 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3210d1f-e314-49c4-b2f0-660612a11d7b admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:39.635908 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3210d1f-e314-49c4-b2f0-660612a11d7b admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:39.640797 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6aeec-0652-42f2-b42a-90202f55a9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:39.641466 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6aeec-0652-42f2-b42a-90202f55a9ee admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:39.641725 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6aeec-0652-42f2-b42a-90202f55a9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:39.642199 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6aeec-0652-42f2-b42a-90202f55a9ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:39.643860 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb6aeec-0652-42f2-b42a-90202f55a9ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:39.645947 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb6aeec-0652-42f2-b42a-90202f55a9ee admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:39.753123 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1191/2380] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:08:39 2022] POST /identity/v3/projects => generated 385 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:08:39.757464 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361eb40a-1d48-415d-81c8-ad1581b2c3f3 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:39.769917 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361eb40a-1d48-415d-81c8-ad1581b2c3f3 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:39.834219 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14d889-4539-4d40-8ee8-ab0a877ebbb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:39.834849 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14d889-4539-4d40-8ee8-ab0a877ebbb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:39.835114 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14d889-4539-4d40-8ee8-ab0a877ebbb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:39.835390 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14d889-4539-4d40-8ee8-ab0a877ebbb6 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:39.838447 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd14d889-4539-4d40-8ee8-ab0a877ebbb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:39.838447 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd14d889-4539-4d40-8ee8-ab0a877ebbb6 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:39.951563 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1190/2381] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:08:39 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:08:39.955677 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6aeec-0652-42f2-b42a-90202f55a9ee admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:39.966083 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6aeec-0652-42f2-b42a-90202f55a9ee admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:39.970761 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22529c8c-0039-4b3e-8ce6-ffdfdcb6c89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:39.971365 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22529c8c-0039-4b3e-8ce6-ffdfdcb6c89e admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:39.971649 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22529c8c-0039-4b3e-8ce6-ffdfdcb6c89e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:39.971927 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22529c8c-0039-4b3e-8ce6-ffdfdcb6c89e admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:39.976588 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22529c8c-0039-4b3e-8ce6-ffdfdcb6c89e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:39.976588 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22529c8c-0039-4b3e-8ce6-ffdfdcb6c89e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:39.976588 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:39.985216 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1192/2382] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:08:39.989866 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14d889-4539-4d40-8ee8-ab0a877ebbb6 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:40.000055 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14d889-4539-4d40-8ee8-ab0a877ebbb6 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:40.004792 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52177908-8c9d-4a2f-843a-b411319860f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:40.006217 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52177908-8c9d-4a2f-843a-b411319860f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:40.006596 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52177908-8c9d-4a2f-843a-b411319860f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:40.006881 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52177908-8c9d-4a2f-843a-b411319860f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:40.007522 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52177908-8c9d-4a2f-843a-b411319860f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:40.009195 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52177908-8c9d-4a2f-843a-b411319860f3 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:40.019643 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1191/2383] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:08:40.023968 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22529c8c-0039-4b3e-8ce6-ffdfdcb6c89e admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:40.035472 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22529c8c-0039-4b3e-8ce6-ffdfdcb6c89e admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:40.040654 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62723558-9fc6-437e-b77f-d9a06459628b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:40.041508 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62723558-9fc6-437e-b77f-d9a06459628b admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:40.041808 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62723558-9fc6-437e-b77f-d9a06459628b admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:40.042099 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62723558-9fc6-437e-b77f-d9a06459628b admin admin] PATH_INFO: `/v3/projects/727112b5c8c9409db00635a6d9d8639b/users/01cd27f3969e41eeb0a727e19dc862db/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:40.057639 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62723558-9fc6-437e-b77f-d9a06459628b admin admin] RBAC: Authorizing `identity:create_grant(project_id=727112b5c8c9409db00635a6d9d8639b, user_id=01cd27f3969e41eeb0a727e19dc862db, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:40.060356 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62723558-9fc6-437e-b77f-d9a06459628b admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:40.171108 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1193/2384] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:08:40 2022] PUT /identity/v3/projects/727112b5c8c9409db00635a6d9d8639b/users/01cd27f3969e41eeb0a727e19dc862db/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:40.183272 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc7b75-8ac7-46e0-adc1-22e35ea430b2 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:40.185539 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc7b75-8ac7-46e0-adc1-22e35ea430b2 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:40.185936 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc7b75-8ac7-46e0-adc1-22e35ea430b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:40.255742 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-a2dc7b75-8ac7-46e0-adc1-22e35ea430b2 None None] MFA Rules not processed for user `01cd27f3969e41eeb0a727e19dc862db`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:40.257112 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a2dc7b75-8ac7-46e0-adc1-22e35ea430b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:40.313390 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:40.334105 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1192/2385] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:40.342194 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62723558-9fc6-437e-b77f-d9a06459628b admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:40.408669 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62723558-9fc6-437e-b77f-d9a06459628b admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:40.416011 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9cbf39-2c31-473a-b84f-0cafb8d74a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:40.416679 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9cbf39-2c31-473a-b84f-0cafb8d74a3d admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:40.417201 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9cbf39-2c31-473a-b84f-0cafb8d74a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:40.417478 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9cbf39-2c31-473a-b84f-0cafb8d74a3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:40.418157 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9cbf39-2c31-473a-b84f-0cafb8d74a3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:40.420045 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9cbf39-2c31-473a-b84f-0cafb8d74a3d admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:40.556930 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1194/2386] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:08:40 2022] POST /identity/v3/projects => generated 385 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:08:40.561082 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dc7b75-8ac7-46e0-adc1-22e35ea430b2 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:40.571842 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dc7b75-8ac7-46e0-adc1-22e35ea430b2 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:40.633529 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae0ce78-d081-43ba-80a7-477c9275ceff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:40.634278 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae0ce78-d081-43ba-80a7-477c9275ceff admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:40.634655 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae0ce78-d081-43ba-80a7-477c9275ceff admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:40.635032 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae0ce78-d081-43ba-80a7-477c9275ceff admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:40.635792 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae0ce78-d081-43ba-80a7-477c9275ceff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:40.638073 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae0ce78-d081-43ba-80a7-477c9275ceff admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:40.768176 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1193/2387] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:08:40 2022] POST /identity/v3/users => generated 299 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:08:40.773006 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9cbf39-2c31-473a-b84f-0cafb8d74a3d admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:40.783861 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9cbf39-2c31-473a-b84f-0cafb8d74a3d admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:40.788503 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987ddba-7bd2-4b1a-b0e1-a87a1740c9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:40.789118 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987ddba-7bd2-4b1a-b0e1-a87a1740c9ec admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:40.789400 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987ddba-7bd2-4b1a-b0e1-a87a1740c9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:40.789677 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987ddba-7bd2-4b1a-b0e1-a87a1740c9ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:40.790325 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d987ddba-7bd2-4b1a-b0e1-a87a1740c9ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:40.791918 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d987ddba-7bd2-4b1a-b0e1-a87a1740c9ec admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:40.803494 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1195/2388] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:08:40.807686 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae0ce78-d081-43ba-80a7-477c9275ceff admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:40.818402 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae0ce78-d081-43ba-80a7-477c9275ceff admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:40.823556 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d060763-5eb0-47c0-b721-27d6a711e50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:40.824198 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d060763-5eb0-47c0-b721-27d6a711e50b admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:40.824506 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d060763-5eb0-47c0-b721-27d6a711e50b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:40.824784 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d060763-5eb0-47c0-b721-27d6a711e50b admin admin] PATH_INFO: `/v3/projects/fca9d6eccf08433f88ef6d5939a62a82/users/b680f1fafdcb4f71b14a3aedba84f931/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:40.840105 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d060763-5eb0-47c0-b721-27d6a711e50b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fca9d6eccf08433f88ef6d5939a62a82, user_id=b680f1fafdcb4f71b14a3aedba84f931, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:40.842833 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d060763-5eb0-47c0-b721-27d6a711e50b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:40.933469 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1194/2389] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:08:40 2022] PUT /identity/v3/projects/fca9d6eccf08433f88ef6d5939a62a82/users/b680f1fafdcb4f71b14a3aedba84f931/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:08:40.937402 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987ddba-7bd2-4b1a-b0e1-a87a1740c9ec admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:40.947828 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987ddba-7bd2-4b1a-b0e1-a87a1740c9ec admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:41.004802 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ecc898-535a-46d3-903e-f1d3f1a93963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:41.005640 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ecc898-535a-46d3-903e-f1d3f1a93963 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:41.005798 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ecc898-535a-46d3-903e-f1d3f1a93963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:41.006076 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ecc898-535a-46d3-903e-f1d3f1a93963 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:41.006718 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ecc898-535a-46d3-903e-f1d3f1a93963 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:41.009209 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ecc898-535a-46d3-903e-f1d3f1a93963 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:41.019458 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1196/2390] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:08:41.023919 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d060763-5eb0-47c0-b721-27d6a711e50b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:41.035070 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d060763-5eb0-47c0-b721-27d6a711e50b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:41.042727 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f88ffa5-2340-4dcf-803f-95a51588c7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:41.043676 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88ffa5-2340-4dcf-803f-95a51588c7c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:41.044119 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88ffa5-2340-4dcf-803f-95a51588c7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:41.044416 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88ffa5-2340-4dcf-803f-95a51588c7c9 admin admin] PATH_INFO: `/v3/projects/fca9d6eccf08433f88ef6d5939a62a82/users/b680f1fafdcb4f71b14a3aedba84f931/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:41.046365 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f88ffa5-2340-4dcf-803f-95a51588c7c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fca9d6eccf08433f88ef6d5939a62a82, user_id=b680f1fafdcb4f71b14a3aedba84f931, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:41.049314 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f88ffa5-2340-4dcf-803f-95a51588c7c9 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:41.067720 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1195/2391] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:08:41 2022] PUT /identity/v3/projects/fca9d6eccf08433f88ef6d5939a62a82/users/b680f1fafdcb4f71b14a3aedba84f931/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:41.080559 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630838db-9b75-4ff4-b1a4-6543d837275d None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:41.080854 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630838db-9b75-4ff4-b1a4-6543d837275d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:41.081137 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630838db-9b75-4ff4-b1a4-6543d837275d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:41.109749 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-630838db-9b75-4ff4-b1a4-6543d837275d None None] MFA Rules not processed for user `b680f1fafdcb4f71b14a3aedba84f931`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:41.221619 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-630838db-9b75-4ff4-b1a4-6543d837275d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:41.221619 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1197/2392] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:41 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:41.225951 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f88ffa5-2340-4dcf-803f-95a51588c7c9 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:41.227699 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6f88ffa5-2340-4dcf-803f-95a51588c7c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:41.389737 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f88ffa5-2340-4dcf-803f-95a51588c7c9 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:41.389737 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f0c8b-4aa2-4b33-8048-7bf799e6230c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:41.389737 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f0c8b-4aa2-4b33-8048-7bf799e6230c service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:41.389737 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f0c8b-4aa2-4b33-8048-7bf799e6230c service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:41.390772 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f0c8b-4aa2-4b33-8048-7bf799e6230c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:41.390772 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f0c8b-4aa2-4b33-8048-7bf799e6230c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:41.390772 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f0c8b-4aa2-4b33-8048-7bf799e6230c service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:41.390772 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1196/2393] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:08:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:08:41.409088 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:41.481775 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630838db-9b75-4ff4-b1a4-6543d837275d None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:41.481775 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-630838db-9b75-4ff4-b1a4-6543d837275d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:41.545559 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630838db-9b75-4ff4-b1a4-6543d837275d None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:41.550020 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a71e0a-7099-41fc-be93-7d995a24082a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:41.550639 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a71e0a-7099-41fc-be93-7d995a24082a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:41.550916 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a71e0a-7099-41fc-be93-7d995a24082a admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:41.551186 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a71e0a-7099-41fc-be93-7d995a24082a admin admin] PATH_INFO: `/v3/users/cfe0e4123d8845d08d9966391ea54bd2` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:41.552433 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a71e0a-7099-41fc-be93-7d995a24082a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfe0e4123d8845d08d9966391ea54bd2)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:41.554382 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a71e0a-7099-41fc-be93-7d995a24082a admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:41.569401 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:41.820222 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-83a71e0a-7099-41fc-be93-7d995a24082a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfe0e4123d8845d08d9966391ea54bd2'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:41.820977 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-83a71e0a-7099-41fc-be93-7d995a24082a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfe0e4123d8845d08d9966391ea54bd2'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:41.933206 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-83a71e0a-7099-41fc-be93-7d995a24082a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfe0e4123d8845d08d9966391ea54bd2'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:41.933939 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-83a71e0a-7099-41fc-be93-7d995a24082a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfe0e4123d8845d08d9966391ea54bd2'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:41.949617 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-83a71e0a-7099-41fc-be93-7d995a24082a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfe0e4123d8845d08d9966391ea54bd2'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:41.963806 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1198/2394] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:08:41 2022] DELETE /identity/v3/users/cfe0e4123d8845d08d9966391ea54bd2 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:41.991063 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ba204-36dc-4066-a291-4adc7d55929d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:41.991063 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ba204-36dc-4066-a291-4adc7d55929d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:41.991063 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ba204-36dc-4066-a291-4adc7d55929d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:42.007908 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-ef6ba204-36dc-4066-a291-4adc7d55929d None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:42.009136 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ef6ba204-36dc-4066-a291-4adc7d55929d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:42.067137 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1197/2395] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:08:42.078058 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a71e0a-7099-41fc-be93-7d995a24082a admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:42.079489 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-83a71e0a-7099-41fc-be93-7d995a24082a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:42.142319 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a71e0a-7099-41fc-be93-7d995a24082a admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:42.147687 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8286bf01-2724-4dfd-9ee8-6e5e2415b59e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:42.149019 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286bf01-2724-4dfd-9ee8-6e5e2415b59e service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:42.149520 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286bf01-2724-4dfd-9ee8-6e5e2415b59e service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:42.149990 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286bf01-2724-4dfd-9ee8-6e5e2415b59e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:42.163638 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8286bf01-2724-4dfd-9ee8-6e5e2415b59e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:42.165833 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8286bf01-2724-4dfd-9ee8-6e5e2415b59e service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:42.176701 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1199/2396] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:08:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:08:42.590471 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:42.694852 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ba204-36dc-4066-a291-4adc7d55929d None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:42.696411 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ef6ba204-36dc-4066-a291-4adc7d55929d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:42.709449 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ba204-36dc-4066-a291-4adc7d55929d None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:42.714592 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3770eca-421e-40a6-a17f-e03f30596a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:42.715232 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3770eca-421e-40a6-a17f-e03f30596a82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:42.715515 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3770eca-421e-40a6-a17f-e03f30596a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:42.715787 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3770eca-421e-40a6-a17f-e03f30596a82 admin admin] PATH_INFO: `/v3/projects/98424e58111945b28f71e09fb5ad573f` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:42.716934 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3770eca-421e-40a6-a17f-e03f30596a82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98424e58111945b28f71e09fb5ad573f)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:42.718865 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3770eca-421e-40a6-a17f-e03f30596a82 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:42.870729 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d3770eca-421e-40a6-a17f-e03f30596a82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98424e58111945b28f71e09fb5ad573f', 'request_id': 'req-d3770eca-421e-40a6-a17f-e03f30596a82'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:42.974999 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d3770eca-421e-40a6-a17f-e03f30596a82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98424e58111945b28f71e09fb5ad573f', 'request_id': 'req-d3770eca-421e-40a6-a17f-e03f30596a82'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:42.983882 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d3770eca-421e-40a6-a17f-e03f30596a82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98424e58111945b28f71e09fb5ad573f', 'request_id': 'req-d3770eca-421e-40a6-a17f-e03f30596a82'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:42.998158 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:43.003424 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d3770eca-421e-40a6-a17f-e03f30596a82 admin admin] The token cache is being invalidate because project 98424e58111945b28f71e09fb5ad573f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:08:43.004232 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d3770eca-421e-40a6-a17f-e03f30596a82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:43.005761 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d3770eca-421e-40a6-a17f-e03f30596a82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:43.012309 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1198/2397] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:08:42 2022] DELETE /identity/v3/projects/98424e58111945b28f71e09fb5ad573f => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:43.013845 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca4b602-0429-4e58-a857-5f33e046c39f None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:43.014125 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca4b602-0429-4e58-a857-5f33e046c39f None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:43.014407 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca4b602-0429-4e58-a857-5f33e046c39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:43.047197 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-cca4b602-0429-4e58-a857-5f33e046c39f None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:43.048841 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-cca4b602-0429-4e58-a857-5f33e046c39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:43.122588 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1200/2398] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:43.130861 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e9f35-357f-43b0-8271-e7e585aea06a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:43.131246 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e9f35-357f-43b0-8271-e7e585aea06a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:43.131528 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e9f35-357f-43b0-8271-e7e585aea06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:43.152016 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-328e9f35-357f-43b0-8271-e7e585aea06a None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:43.153280 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-328e9f35-357f-43b0-8271-e7e585aea06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:43.165878 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1199/2399] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:43.170183 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca4b602-0429-4e58-a857-5f33e046c39f None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:43.180347 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca4b602-0429-4e58-a857-5f33e046c39f None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:43.185616 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b010000-db29-4921-b63d-385c62d7a7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:43.186286 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b010000-db29-4921-b63d-385c62d7a7e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:43.186567 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b010000-db29-4921-b63d-385c62d7a7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:43.186837 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b010000-db29-4921-b63d-385c62d7a7e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:43.187488 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b010000-db29-4921-b63d-385c62d7a7e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:08:43.187786 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b010000-db29-4921-b63d-385c62d7a7e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:43.189481 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b010000-db29-4921-b63d-385c62d7a7e2 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:43.202041 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1201/2400] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:08:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 01:08:43.206451 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e9f35-357f-43b0-8271-e7e585aea06a None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:43.216223 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e9f35-357f-43b0-8271-e7e585aea06a None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:43.221480 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c1dd6-50f3-4470-854d-fe82d71c57e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:43.222274 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c1dd6-50f3-4470-854d-fe82d71c57e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:43.222715 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c1dd6-50f3-4470-854d-fe82d71c57e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:43.223136 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c1dd6-50f3-4470-854d-fe82d71c57e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:43.224695 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352c1dd6-50f3-4470-854d-fe82d71c57e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:43.228183 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352c1dd6-50f3-4470-854d-fe82d71c57e0 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:43.336198 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1200/2401] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:08:43 2022] POST /identity/v3/projects => generated 383 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:08:43.337222 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b010000-db29-4921-b63d-385c62d7a7e2 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:43.348016 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b010000-db29-4921-b63d-385c62d7a7e2 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:43.409893 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762715d5-0e8c-437e-9e82-db5c05604a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:43.410511 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762715d5-0e8c-437e-9e82-db5c05604a29 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:43.410792 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762715d5-0e8c-437e-9e82-db5c05604a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:43.411262 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762715d5-0e8c-437e-9e82-db5c05604a29 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:43.412009 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762715d5-0e8c-437e-9e82-db5c05604a29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:43.414004 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762715d5-0e8c-437e-9e82-db5c05604a29 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:43.465153 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195f414-5173-4421-88b4-5165a3f1cd8b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:43.465544 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195f414-5173-4421-88b4-5165a3f1cd8b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:43.465889 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195f414-5173-4421-88b4-5165a3f1cd8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:43.485008 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-b195f414-5173-4421-88b4-5165a3f1cd8b None None] MFA Rules not processed for user `9bef8dc1da4c44dbbafcaf3c813b3262`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:43.486145 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b195f414-5173-4421-88b4-5165a3f1cd8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:43.530418 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1202/2402] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:08:43 2022] POST /identity/v3/users => generated 292 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:08:43.531326 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762715d5-0e8c-437e-9e82-db5c05604a29 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:43.547017 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762715d5-0e8c-437e-9e82-db5c05604a29 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:43.557853 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1201/2403] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:43 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:08:43.560007 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa4ee2e-584c-47b4-abfc-a5ad25f53486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:43.560879 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4ee2e-584c-47b4-abfc-a5ad25f53486 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:43.561410 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4ee2e-584c-47b4-abfc-a5ad25f53486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:43.561890 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4ee2e-584c-47b4-abfc-a5ad25f53486 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:43.563124 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa4ee2e-584c-47b4-abfc-a5ad25f53486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:43.565180 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa4ee2e-584c-47b4-abfc-a5ad25f53486 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:43.579744 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1203/2404] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 01:08:43.589315 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195f414-5173-4421-88b4-5165a3f1cd8b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:43.601837 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195f414-5173-4421-88b4-5165a3f1cd8b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:43.608175 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2568e9-566d-4fd6-b5b9-7b1dedd9c8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:43.608175 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2568e9-566d-4fd6-b5b9-7b1dedd9c8ce admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:43.608175 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2568e9-566d-4fd6-b5b9-7b1dedd9c8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:43.608175 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2568e9-566d-4fd6-b5b9-7b1dedd9c8ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:43.608624 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2568e9-566d-4fd6-b5b9-7b1dedd9c8ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:43.610140 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2568e9-566d-4fd6-b5b9-7b1dedd9c8ce admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:43.622656 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1202/2405] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:08:43.626885 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa4ee2e-584c-47b4-abfc-a5ad25f53486 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:43.638865 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa4ee2e-584c-47b4-abfc-a5ad25f53486 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:43.643542 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c264641b-13fd-4696-9d3d-025c19c567e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:43.644205 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264641b-13fd-4696-9d3d-025c19c567e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:43.644513 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264641b-13fd-4696-9d3d-025c19c567e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:43.644795 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264641b-13fd-4696-9d3d-025c19c567e3 admin admin] PATH_INFO: `/v3/projects/18076036cce84d48bc9df44f5b563bac/users/6ddda0a1a0774b2da6bc75cbb10e094b/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:43.662879 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c264641b-13fd-4696-9d3d-025c19c567e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18076036cce84d48bc9df44f5b563bac, user_id=6ddda0a1a0774b2da6bc75cbb10e094b, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:43.666212 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c264641b-13fd-4696-9d3d-025c19c567e3 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:43.754454 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1204/2406] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:08:43 2022] PUT /identity/v3/projects/18076036cce84d48bc9df44f5b563bac/users/6ddda0a1a0774b2da6bc75cbb10e094b/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:43.761910 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b1d51-07aa-4d7f-8785-f489f1ccdd61 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:43.762288 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b1d51-07aa-4d7f-8785-f489f1ccdd61 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:43.762669 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b1d51-07aa-4d7f-8785-f489f1ccdd61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:43.786544 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f43b1d51-07aa-4d7f-8785-f489f1ccdd61 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:43.787811 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f43b1d51-07aa-4d7f-8785-f489f1ccdd61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:43.855078 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1203/2407] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:43.865592 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c264641b-13fd-4696-9d3d-025c19c567e3 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:43.868957 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c264641b-13fd-4696-9d3d-025c19c567e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:43.904821 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:43.928051 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c264641b-13fd-4696-9d3d-025c19c567e3 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:43.931728 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2586bf10-dd58-4ffe-9edf-83e0ccc9d27d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:43.932334 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586bf10-dd58-4ffe-9edf-83e0ccc9d27d service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:43.932647 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586bf10-dd58-4ffe-9edf-83e0ccc9d27d service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:43.932919 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586bf10-dd58-4ffe-9edf-83e0ccc9d27d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:43.942078 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2586bf10-dd58-4ffe-9edf-83e0ccc9d27d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:43.944874 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2586bf10-dd58-4ffe-9edf-83e0ccc9d27d service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:43.955098 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1205/2408] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:08:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:08:44.677110 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43b1d51-07aa-4d7f-8785-f489f1ccdd61 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:44.678673 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f43b1d51-07aa-4d7f-8785-f489f1ccdd61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:44.744105 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43b1d51-07aa-4d7f-8785-f489f1ccdd61 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:44.748111 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47258fe1-31f9-46ca-80c3-1b23d348974c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:44.749000 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47258fe1-31f9-46ca-80c3-1b23d348974c service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:44.749507 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47258fe1-31f9-46ca-80c3-1b23d348974c service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:44.749798 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47258fe1-31f9-46ca-80c3-1b23d348974c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:44.752036 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-47258fe1-31f9-46ca-80c3-1b23d348974c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:44.783528 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:44.827092 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47258fe1-31f9-46ca-80c3-1b23d348974c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:44.828623 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47258fe1-31f9-46ca-80c3-1b23d348974c service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:44.840275 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1204/2409] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:08:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:08:45.312219 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f1794-88c4-41c6-ad77-74ec05287d32 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:45.313395 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f1794-88c4-41c6-ad77-74ec05287d32 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:45.313857 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f1794-88c4-41c6-ad77-74ec05287d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:45.347931 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-811f1794-88c4-41c6-ad77-74ec05287d32 None None] MFA Rules not processed for user `6ddda0a1a0774b2da6bc75cbb10e094b`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:45.350018 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-811f1794-88c4-41c6-ad77-74ec05287d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:45.433448 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1206/2410] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:45.469371 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47258fe1-31f9-46ca-80c3-1b23d348974c service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:45.471068 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-47258fe1-31f9-46ca-80c3-1b23d348974c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:45.482522 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47258fe1-31f9-46ca-80c3-1b23d348974c service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:45.487247 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af10462-c733-463d-a475-936d50319b1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:45.488078 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af10462-c733-463d-a475-936d50319b1c service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:45.488590 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af10462-c733-463d-a475-936d50319b1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:45.490280 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af10462-c733-463d-a475-936d50319b1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:45.499133 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af10462-c733-463d-a475-936d50319b1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:45.500709 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af10462-c733-463d-a475-936d50319b1c service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:45.510815 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1205/2411] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:08:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:08:46.415960 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f1794-88c4-41c6-ad77-74ec05287d32 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:46.425706 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f1794-88c4-41c6-ad77-74ec05287d32 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:46.430175 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d35c3-e460-42f7-accf-8be3b0a7338b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:46.430778 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d35c3-e460-42f7-accf-8be3b0a7338b service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:46.431063 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d35c3-e460-42f7-accf-8be3b0a7338b service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:46.431342 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d35c3-e460-42f7-accf-8be3b0a7338b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:46.433149 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-0b0d35c3-e460-42f7-accf-8be3b0a7338b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:46.500030 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d35c3-e460-42f7-accf-8be3b0a7338b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:46.503897 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d35c3-e460-42f7-accf-8be3b0a7338b service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:46.514134 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1207/2412] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:08:46 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:08:47.583175 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af10462-c733-463d-a475-936d50319b1c service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:47.584727 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5af10462-c733-463d-a475-936d50319b1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:47.599391 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af10462-c733-463d-a475-936d50319b1c service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:47.604884 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70279c30-fa62-408a-8e43-34c21ca776cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:47.607404 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70279c30-fa62-408a-8e43-34c21ca776cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:47.607404 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70279c30-fa62-408a-8e43-34c21ca776cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:47.607404 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70279c30-fa62-408a-8e43-34c21ca776cf admin admin] PATH_INFO: `/v3/users/9af5aec0e67f41d6b356af7859708b85` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:47.607404 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70279c30-fa62-408a-8e43-34c21ca776cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=9af5aec0e67f41d6b356af7859708b85)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:47.609253 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70279c30-fa62-408a-8e43-34c21ca776cf admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:47.829655 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-70279c30-fa62-408a-8e43-34c21ca776cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9af5aec0e67f41d6b356af7859708b85'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:47.830609 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-70279c30-fa62-408a-8e43-34c21ca776cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9af5aec0e67f41d6b356af7859708b85'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:47.839553 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-70279c30-fa62-408a-8e43-34c21ca776cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9af5aec0e67f41d6b356af7859708b85'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:47.911412 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-70279c30-fa62-408a-8e43-34c21ca776cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9af5aec0e67f41d6b356af7859708b85'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:47.912350 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-70279c30-fa62-408a-8e43-34c21ca776cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9af5aec0e67f41d6b356af7859708b85'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:47.935887 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1206/2413] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:08:47 2022] DELETE /identity/v3/users/9af5aec0e67f41d6b356af7859708b85 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:47.940658 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003cb567-7495-4983-9ad5-11ac034c1b10 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:47.941698 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003cb567-7495-4983-9ad5-11ac034c1b10 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:47.942684 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003cb567-7495-4983-9ad5-11ac034c1b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:47.964389 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-003cb567-7495-4983-9ad5-11ac034c1b10 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:47.968136 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-003cb567-7495-4983-9ad5-11ac034c1b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:48.020733 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:48.029009 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1208/2414] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:08:48.038880 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70279c30-fa62-408a-8e43-34c21ca776cf admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:48.040348 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-70279c30-fa62-408a-8e43-34c21ca776cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:48.082297 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003cb567-7495-4983-9ad5-11ac034c1b10 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:48.084188 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-003cb567-7495-4983-9ad5-11ac034c1b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:48.104197 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003cb567-7495-4983-9ad5-11ac034c1b10 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:48.109821 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c25774-b9e6-4eaf-9749-5e930e8b0ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:48.111676 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c25774-b9e6-4eaf-9749-5e930e8b0ead admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:48.111676 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c25774-b9e6-4eaf-9749-5e930e8b0ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:48.111676 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c25774-b9e6-4eaf-9749-5e930e8b0ead admin admin] PATH_INFO: `/v3/users/7a1fc708e8cd4b42baa8a661a625606b` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:48.112523 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c25774-b9e6-4eaf-9749-5e930e8b0ead admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a1fc708e8cd4b42baa8a661a625606b)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:48.114824 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c25774-b9e6-4eaf-9749-5e930e8b0ead admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:48.152814 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70279c30-fa62-408a-8e43-34c21ca776cf admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:48.157320 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30bf970-0164-4ba8-b7c9-124bb9bcbcde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:48.157947 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bf970-0164-4ba8-b7c9-124bb9bcbcde service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:48.158232 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bf970-0164-4ba8-b7c9-124bb9bcbcde service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:48.158565 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bf970-0164-4ba8-b7c9-124bb9bcbcde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:48.169129 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30bf970-0164-4ba8-b7c9-124bb9bcbcde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:48.170574 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30bf970-0164-4ba8-b7c9-124bb9bcbcde service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:48.181962 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1207/2415] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:08:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:08:48.338041 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-55c25774-b9e6-4eaf-9749-5e930e8b0ead admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a1fc708e8cd4b42baa8a661a625606b'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:48.338711 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-55c25774-b9e6-4eaf-9749-5e930e8b0ead admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a1fc708e8cd4b42baa8a661a625606b'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:48.424113 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-55c25774-b9e6-4eaf-9749-5e930e8b0ead admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a1fc708e8cd4b42baa8a661a625606b'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:48.424352 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f435a8f-67e9-4fdd-80e6-c4cddde93735 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:48.424618 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f435a8f-67e9-4fdd-80e6-c4cddde93735 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:48.425004 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f435a8f-67e9-4fdd-80e6-c4cddde93735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:48.427245 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-55c25774-b9e6-4eaf-9749-5e930e8b0ead admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a1fc708e8cd4b42baa8a661a625606b'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:48.449292 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-55c25774-b9e6-4eaf-9749-5e930e8b0ead admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a1fc708e8cd4b42baa8a661a625606b'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:48.491037 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1209/2416] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:08:48 2022] DELETE /identity/v3/users/7a1fc708e8cd4b42baa8a661a625606b => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:08:48.496619 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-6f435a8f-67e9-4fdd-80e6-c4cddde93735 None None] MFA Rules not processed for user `6ddda0a1a0774b2da6bc75cbb10e094b`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:48.497869 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6f435a8f-67e9-4fdd-80e6-c4cddde93735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:48.529172 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:48.570905 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1208/2417] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:48.781686 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c25774-b9e6-4eaf-9749-5e930e8b0ead admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:48.783357 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-55c25774-b9e6-4eaf-9749-5e930e8b0ead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:48.879703 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c25774-b9e6-4eaf-9749-5e930e8b0ead admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:48.886511 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f435a8f-67e9-4fdd-80e6-c4cddde93735 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:48.888407 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6f435a8f-67e9-4fdd-80e6-c4cddde93735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:48.890054 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66f08e5-e231-4c5a-9699-1714ecc87efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:48.890822 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f08e5-e231-4c5a-9699-1714ecc87efe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:48.891174 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f08e5-e231-4c5a-9699-1714ecc87efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:48.891559 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f08e5-e231-4c5a-9699-1714ecc87efe admin admin] PATH_INFO: `/v3/projects/54588287e5e440c9ba341826fa75870f` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:48.893006 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66f08e5-e231-4c5a-9699-1714ecc87efe admin admin] RBAC: Authorizing `identity:delete_project(project_id=54588287e5e440c9ba341826fa75870f)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:48.895745 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66f08e5-e231-4c5a-9699-1714ecc87efe admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:48.907385 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f435a8f-67e9-4fdd-80e6-c4cddde93735 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:48.916218 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a837e89d-b435-45d0-a46f-dd5607f6c41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:48.917236 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837e89d-b435-45d0-a46f-dd5607f6c41b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:48.917574 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837e89d-b435-45d0-a46f-dd5607f6c41b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:48.917853 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837e89d-b435-45d0-a46f-dd5607f6c41b admin admin] PATH_INFO: `/v3/projects/3ece345f5fb34b3cbdb32d2880bf67da` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:48.919018 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a837e89d-b435-45d0-a46f-dd5607f6c41b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ece345f5fb34b3cbdb32d2880bf67da)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:48.923751 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a837e89d-b435-45d0-a46f-dd5607f6c41b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:49.041081 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a837e89d-b435-45d0-a46f-dd5607f6c41b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ece345f5fb34b3cbdb32d2880bf67da', 'request_id': 'req-a837e89d-b435-45d0-a46f-dd5607f6c41b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:49.070842 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e66f08e5-e231-4c5a-9699-1714ecc87efe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54588287e5e440c9ba341826fa75870f', 'request_id': 'req-e66f08e5-e231-4c5a-9699-1714ecc87efe'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:49.128610 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a837e89d-b435-45d0-a46f-dd5607f6c41b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ece345f5fb34b3cbdb32d2880bf67da', 'request_id': 'req-a837e89d-b435-45d0-a46f-dd5607f6c41b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:49.136588 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a837e89d-b435-45d0-a46f-dd5607f6c41b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ece345f5fb34b3cbdb32d2880bf67da', 'request_id': 'req-a837e89d-b435-45d0-a46f-dd5607f6c41b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:49.151208 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a837e89d-b435-45d0-a46f-dd5607f6c41b admin admin] The token cache is being invalidate because project 3ece345f5fb34b3cbdb32d2880bf67da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:08:49.151456 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a837e89d-b435-45d0-a46f-dd5607f6c41b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:49.152126 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a837e89d-b435-45d0-a46f-dd5607f6c41b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:49.153931 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1209/2418] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:08:48 2022] DELETE /identity/v3/projects/3ece345f5fb34b3cbdb32d2880bf67da => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:49.167677 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a837e89d-b435-45d0-a46f-dd5607f6c41b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:49.173298 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a837e89d-b435-45d0-a46f-dd5607f6c41b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:49.213955 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e66f08e5-e231-4c5a-9699-1714ecc87efe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54588287e5e440c9ba341826fa75870f', 'request_id': 'req-e66f08e5-e231-4c5a-9699-1714ecc87efe'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:49.221205 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e66f08e5-e231-4c5a-9699-1714ecc87efe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54588287e5e440c9ba341826fa75870f', 'request_id': 'req-e66f08e5-e231-4c5a-9699-1714ecc87efe'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:49.237591 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e66f08e5-e231-4c5a-9699-1714ecc87efe admin admin] The token cache is being invalidate because project 54588287e5e440c9ba341826fa75870f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:08:49.237882 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e66f08e5-e231-4c5a-9699-1714ecc87efe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:49.238550 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e66f08e5-e231-4c5a-9699-1714ecc87efe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:49.240880 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1210/2419] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:08:48 2022] DELETE /identity/v3/projects/54588287e5e440c9ba341826fa75870f => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:49.246160 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f089be4-6eb3-46b6-a677-9ef686fcdf8d None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:49.251349 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f089be4-6eb3-46b6-a677-9ef686fcdf8d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:49.251349 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f089be4-6eb3-46b6-a677-9ef686fcdf8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:49.251989 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a837e89d-b435-45d0-a46f-dd5607f6c41b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:49.255205 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:49.268132 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-9f089be4-6eb3-46b6-a677-9ef686fcdf8d None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:49.269402 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-9f089be4-6eb3-46b6-a677-9ef686fcdf8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:49.351332 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bcbb77-4c9f-4046-aa27-5082e89a2106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:49.351996 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bcbb77-4c9f-4046-aa27-5082e89a2106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:49.353656 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bcbb77-4c9f-4046-aa27-5082e89a2106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:49.353965 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bcbb77-4c9f-4046-aa27-5082e89a2106 admin admin] PATH_INFO: `/v3/users/1ab9ad9da6af48949165a04171a9adc2` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:49.358721 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bcbb77-4c9f-4046-aa27-5082e89a2106 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ab9ad9da6af48949165a04171a9adc2)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:49.363963 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bcbb77-4c9f-4046-aa27-5082e89a2106 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:49.400493 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1211/2420] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:08:49.411101 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae57612e-afa5-45f6-900b-e3702838598b None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:49.411715 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae57612e-afa5-45f6-900b-e3702838598b None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:49.412143 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae57612e-afa5-45f6-900b-e3702838598b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:49.438179 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-ae57612e-afa5-45f6-900b-e3702838598b None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:49.438179 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ae57612e-afa5-45f6-900b-e3702838598b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:49.450541 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1212/2421] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:49.460538 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae57612e-afa5-45f6-900b-e3702838598b None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:49.465931 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae57612e-afa5-45f6-900b-e3702838598b None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:49.472968 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe6ca8-8be2-455a-aba6-86ec050f7ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:49.473844 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe6ca8-8be2-455a-aba6-86ec050f7ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:49.475884 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe6ca8-8be2-455a-aba6-86ec050f7ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:49.476118 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe6ca8-8be2-455a-aba6-86ec050f7ca1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:49.476822 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe6ca8-8be2-455a-aba6-86ec050f7ca1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:08:49.477131 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe6ca8-8be2-455a-aba6-86ec050f7ca1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:49.478787 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe6ca8-8be2-455a-aba6-86ec050f7ca1 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:49.496278 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1213/2422] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:08:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:08:49.503031 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe6ca8-8be2-455a-aba6-86ec050f7ca1 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:49.516621 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe6ca8-8be2-455a-aba6-86ec050f7ca1 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:49.520915 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744db842-b7da-4641-9ec9-eba27f7bcf67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:49.521667 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744db842-b7da-4641-9ec9-eba27f7bcf67 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:49.522025 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744db842-b7da-4641-9ec9-eba27f7bcf67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:49.522309 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744db842-b7da-4641-9ec9-eba27f7bcf67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:49.523285 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744db842-b7da-4641-9ec9-eba27f7bcf67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:49.525702 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744db842-b7da-4641-9ec9-eba27f7bcf67 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:49.527050 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:49.632169 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-53bcbb77-4c9f-4046-aa27-5082e89a2106 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ab9ad9da6af48949165a04171a9adc2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:49.632848 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-53bcbb77-4c9f-4046-aa27-5082e89a2106 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ab9ad9da6af48949165a04171a9adc2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:49.640385 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-53bcbb77-4c9f-4046-aa27-5082e89a2106 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ab9ad9da6af48949165a04171a9adc2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:49.653293 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1214/2423] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:08:49 2022] POST /identity/v3/projects => generated 371 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:08:49.657444 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744db842-b7da-4641-9ec9-eba27f7bcf67 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:49.662011 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-744db842-b7da-4641-9ec9-eba27f7bcf67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:49.739639 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744db842-b7da-4641-9ec9-eba27f7bcf67 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:49.744812 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1125fd-7047-4183-9d4e-8ff80917a13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:49.745430 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1125fd-7047-4183-9d4e-8ff80917a13c admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:49.745979 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1125fd-7047-4183-9d4e-8ff80917a13c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:49.746268 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1125fd-7047-4183-9d4e-8ff80917a13c admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:49.746950 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1125fd-7047-4183-9d4e-8ff80917a13c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:49.748982 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1125fd-7047-4183-9d4e-8ff80917a13c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:49.757339 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-53bcbb77-4c9f-4046-aa27-5082e89a2106 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ab9ad9da6af48949165a04171a9adc2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:49.759464 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-53bcbb77-4c9f-4046-aa27-5082e89a2106 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ab9ad9da6af48949165a04171a9adc2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:49.779896 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1210/2424] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:08:49 2022] DELETE /identity/v3/users/1ab9ad9da6af48949165a04171a9adc2 => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:49.851433 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1215/2425] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:08:49 2022] POST /identity/v3/users => generated 286 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:08:49.852838 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bcbb77-4c9f-4046-aa27-5082e89a2106 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:49.925244 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bcbb77-4c9f-4046-aa27-5082e89a2106 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:49.930821 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e2961-18ee-4898-8c13-43a1a4bb7629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:49.935195 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e2961-18ee-4898-8c13-43a1a4bb7629 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:49.935195 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e2961-18ee-4898-8c13-43a1a4bb7629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:49.935195 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e2961-18ee-4898-8c13-43a1a4bb7629 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:49.935195 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613e2961-18ee-4898-8c13-43a1a4bb7629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:49.935195 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613e2961-18ee-4898-8c13-43a1a4bb7629 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:49.951919 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1211/2426] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:08:49.953441 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1125fd-7047-4183-9d4e-8ff80917a13c admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:49.964896 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1125fd-7047-4183-9d4e-8ff80917a13c admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:49.969682 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca50021-0839-4678-8ca2-1279e015945d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:49.970567 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca50021-0839-4678-8ca2-1279e015945d admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:49.970844 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca50021-0839-4678-8ca2-1279e015945d admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:49.971153 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca50021-0839-4678-8ca2-1279e015945d admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:49.971817 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca50021-0839-4678-8ca2-1279e015945d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:49.974448 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca50021-0839-4678-8ca2-1279e015945d admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:49.985491 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1216/2427] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:08:49.990757 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e2961-18ee-4898-8c13-43a1a4bb7629 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:50.001247 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e2961-18ee-4898-8c13-43a1a4bb7629 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:50.007291 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c641cee-3ce0-4bcb-a270-72a87361fa32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:50.008649 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c641cee-3ce0-4bcb-a270-72a87361fa32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:50.009318 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c641cee-3ce0-4bcb-a270-72a87361fa32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:50.009804 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c641cee-3ce0-4bcb-a270-72a87361fa32 admin admin] PATH_INFO: `/v3/projects/be21c8fcaef64d9dbecf31bd633559c4/users/7fee1d5bc3da421fa1c85ea59223fb54/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:50.028242 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c641cee-3ce0-4bcb-a270-72a87361fa32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be21c8fcaef64d9dbecf31bd633559c4, user_id=7fee1d5bc3da421fa1c85ea59223fb54, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:50.031180 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c641cee-3ce0-4bcb-a270-72a87361fa32 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:50.102353 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1212/2428] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:08:49 2022] PUT /identity/v3/projects/be21c8fcaef64d9dbecf31bd633559c4/users/7fee1d5bc3da421fa1c85ea59223fb54/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:50.107717 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e418ed-95e7-4559-bdde-19ae0f379938 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:50.108192 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e418ed-95e7-4559-bdde-19ae0f379938 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:50.108288 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e418ed-95e7-4559-bdde-19ae0f379938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:50.129000 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-35e418ed-95e7-4559-bdde-19ae0f379938 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:50.130935 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-35e418ed-95e7-4559-bdde-19ae0f379938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:50.201506 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1217/2429] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:50.215966 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e418ed-95e7-4559-bdde-19ae0f379938 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:50.216103 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c641cee-3ce0-4bcb-a270-72a87361fa32 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:50.217383 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-35e418ed-95e7-4559-bdde-19ae0f379938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:50.217771 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8c641cee-3ce0-4bcb-a270-72a87361fa32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:50.240052 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e418ed-95e7-4559-bdde-19ae0f379938 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:50.246560 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6a47dc-285d-40ca-94fb-d133491a4243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:50.247209 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a47dc-285d-40ca-94fb-d133491a4243 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:50.247488 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a47dc-285d-40ca-94fb-d133491a4243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:50.247749 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a47dc-285d-40ca-94fb-d133491a4243 admin admin] PATH_INFO: `/v3/projects/389e0e44d28649d5bd12db42a8312f0e` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:50.250076 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6a47dc-285d-40ca-94fb-d133491a4243 admin admin] RBAC: Authorizing `identity:delete_project(project_id=389e0e44d28649d5bd12db42a8312f0e)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:50.251592 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6a47dc-285d-40ca-94fb-d133491a4243 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:50.311684 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c641cee-3ce0-4bcb-a270-72a87361fa32 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:50.319855 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b964f-162f-4d4e-ba4d-3729d9cc6bbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:50.321217 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b964f-162f-4d4e-ba4d-3729d9cc6bbb service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:50.321926 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b964f-162f-4d4e-ba4d-3729d9cc6bbb service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:50.322209 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b964f-162f-4d4e-ba4d-3729d9cc6bbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:50.333093 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8b964f-162f-4d4e-ba4d-3729d9cc6bbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:50.334272 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8b964f-162f-4d4e-ba4d-3729d9cc6bbb service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:50.344908 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1213/2430] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:08:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:08:50.446525 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2d6a47dc-285d-40ca-94fb-d133491a4243 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '389e0e44d28649d5bd12db42a8312f0e', 'request_id': 'req-2d6a47dc-285d-40ca-94fb-d133491a4243'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:50.517224 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2d6a47dc-285d-40ca-94fb-d133491a4243 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '389e0e44d28649d5bd12db42a8312f0e', 'request_id': 'req-2d6a47dc-285d-40ca-94fb-d133491a4243'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:50.524749 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2d6a47dc-285d-40ca-94fb-d133491a4243 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '389e0e44d28649d5bd12db42a8312f0e', 'request_id': 'req-2d6a47dc-285d-40ca-94fb-d133491a4243'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:50.549411 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2d6a47dc-285d-40ca-94fb-d133491a4243 admin admin] The token cache is being invalidate because project 389e0e44d28649d5bd12db42a8312f0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:08:50.549790 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2d6a47dc-285d-40ca-94fb-d133491a4243 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:50.550540 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2d6a47dc-285d-40ca-94fb-d133491a4243 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:50.552934 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1218/2431] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:08:50 2022] DELETE /identity/v3/projects/389e0e44d28649d5bd12db42a8312f0e => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:50.558168 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553110ea-f0c7-4a0f-a252-4f2f6f9efefc None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:50.558533 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553110ea-f0c7-4a0f-a252-4f2f6f9efefc None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:50.559002 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553110ea-f0c7-4a0f-a252-4f2f6f9efefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:50.588922 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-553110ea-f0c7-4a0f-a252-4f2f6f9efefc None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:50.590276 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-553110ea-f0c7-4a0f-a252-4f2f6f9efefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:50.658297 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1214/2432] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:08:50.660348 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:50.666561 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ab2d5-5c7e-451a-82b4-4efd94e2c742 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:50.668283 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ab2d5-5c7e-451a-82b4-4efd94e2c742 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:50.668283 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ab2d5-5c7e-451a-82b4-4efd94e2c742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:50.701453 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-567ab2d5-5c7e-451a-82b4-4efd94e2c742 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:50.701453 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-567ab2d5-5c7e-451a-82b4-4efd94e2c742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:50.708629 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1219/2433] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:50.713025 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553110ea-f0c7-4a0f-a252-4f2f6f9efefc None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:50.723708 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553110ea-f0c7-4a0f-a252-4f2f6f9efefc None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:50.728547 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c999466-2714-44e1-8cda-cbe86069875c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:50.729159 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c999466-2714-44e1-8cda-cbe86069875c admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:50.729443 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c999466-2714-44e1-8cda-cbe86069875c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:50.729718 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c999466-2714-44e1-8cda-cbe86069875c admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:50.730425 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c999466-2714-44e1-8cda-cbe86069875c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:08:50.730875 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c999466-2714-44e1-8cda-cbe86069875c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:50.732295 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c999466-2714-44e1-8cda-cbe86069875c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:50.745465 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1215/2434] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:08:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:08:50.749947 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567ab2d5-5c7e-451a-82b4-4efd94e2c742 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:50.760043 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567ab2d5-5c7e-451a-82b4-4efd94e2c742 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:50.773589 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c1a8b-ce7d-4ce0-a561-6c05c6b45e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:50.774887 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c1a8b-ce7d-4ce0-a561-6c05c6b45e5b admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:50.775242 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c1a8b-ce7d-4ce0-a561-6c05c6b45e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:50.775529 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c1a8b-ce7d-4ce0-a561-6c05c6b45e5b admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:50.776237 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8c1a8b-ce7d-4ce0-a561-6c05c6b45e5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:50.778400 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8c1a8b-ce7d-4ce0-a561-6c05c6b45e5b admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:50.905053 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1220/2435] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:08:50 2022] POST /identity/v3/projects => generated 383 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:08:50.909406 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c999466-2714-44e1-8cda-cbe86069875c admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:50.919979 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c999466-2714-44e1-8cda-cbe86069875c admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:50.922632 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:50.984281 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aec3425-5a36-464c-b235-9d8ee2e98228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:50.985065 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec3425-5a36-464c-b235-9d8ee2e98228 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:50.985293 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec3425-5a36-464c-b235-9d8ee2e98228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:50.985572 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec3425-5a36-464c-b235-9d8ee2e98228 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:50.986207 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aec3425-5a36-464c-b235-9d8ee2e98228 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:50.988157 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aec3425-5a36-464c-b235-9d8ee2e98228 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:51.085840 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1216/2436] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:08:50 2022] POST /identity/v3/users => generated 292 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:08:51.086482 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c1a8b-ce7d-4ce0-a561-6c05c6b45e5b admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:51.105060 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c1a8b-ce7d-4ce0-a561-6c05c6b45e5b admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:51.118770 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aaa7d0-0ef7-45bc-8b62-32c215585a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:51.119425 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aaa7d0-0ef7-45bc-8b62-32c215585a65 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:51.119666 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aaa7d0-0ef7-45bc-8b62-32c215585a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:51.119928 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aaa7d0-0ef7-45bc-8b62-32c215585a65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:51.120608 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aaa7d0-0ef7-45bc-8b62-32c215585a65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:51.122221 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aaa7d0-0ef7-45bc-8b62-32c215585a65 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:51.134192 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1221/2437] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:51 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:08:51.142901 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aec3425-5a36-464c-b235-9d8ee2e98228 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:51.153168 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aec3425-5a36-464c-b235-9d8ee2e98228 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:51.159204 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dad6c1d-4e7a-442c-9972-7ca46f4439c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:51.159806 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad6c1d-4e7a-442c-9972-7ca46f4439c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:51.160168 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad6c1d-4e7a-442c-9972-7ca46f4439c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:51.160338 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad6c1d-4e7a-442c-9972-7ca46f4439c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:51.161031 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dad6c1d-4e7a-442c-9972-7ca46f4439c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:51.162988 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dad6c1d-4e7a-442c-9972-7ca46f4439c8 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:51.174114 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1217/2438] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:08:51.178915 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aaa7d0-0ef7-45bc-8b62-32c215585a65 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:51.189226 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aaa7d0-0ef7-45bc-8b62-32c215585a65 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:51.195312 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055dfd04-d9f3-468f-9f4d-21c1ef324a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:51.195973 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055dfd04-d9f3-468f-9f4d-21c1ef324a49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:51.196259 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055dfd04-d9f3-468f-9f4d-21c1ef324a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:51.196569 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055dfd04-d9f3-468f-9f4d-21c1ef324a49 admin admin] PATH_INFO: `/v3/projects/4715bba986c643c285f8cdf105d44a23/users/9e1177488c17428eb8286eea3a2b216e/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:51.211761 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055dfd04-d9f3-468f-9f4d-21c1ef324a49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4715bba986c643c285f8cdf105d44a23, user_id=9e1177488c17428eb8286eea3a2b216e, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:51.214659 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055dfd04-d9f3-468f-9f4d-21c1ef324a49 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:51.300281 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1222/2439] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:08:51 2022] PUT /identity/v3/projects/4715bba986c643c285f8cdf105d44a23/users/9e1177488c17428eb8286eea3a2b216e/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:51.311561 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a54c3e-2c1e-4488-99ad-de596999a965 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:51.311846 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a54c3e-2c1e-4488-99ad-de596999a965 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:51.312120 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a54c3e-2c1e-4488-99ad-de596999a965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:51.342060 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-44a54c3e-2c1e-4488-99ad-de596999a965 None None] MFA Rules not processed for user `9e1177488c17428eb8286eea3a2b216e`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:51.344713 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-44a54c3e-2c1e-4488-99ad-de596999a965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:51.417300 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1218/2440] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:51.429087 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055dfd04-d9f3-468f-9f4d-21c1ef324a49 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:51.431023 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-055dfd04-d9f3-468f-9f4d-21c1ef324a49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:51.489413 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055dfd04-d9f3-468f-9f4d-21c1ef324a49 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:51.493373 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa03c3c-bdf1-4b4d-8b08-1b39180627e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:51.494055 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa03c3c-bdf1-4b4d-8b08-1b39180627e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:51.494362 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa03c3c-bdf1-4b4d-8b08-1b39180627e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:51.494640 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa03c3c-bdf1-4b4d-8b08-1b39180627e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:51.503232 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa03c3c-bdf1-4b4d-8b08-1b39180627e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:51.504760 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa03c3c-bdf1-4b4d-8b08-1b39180627e3 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:51.513743 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1223/2441] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:08:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:08:52.503353 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a54c3e-2c1e-4488-99ad-de596999a965 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:52.505630 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-44a54c3e-2c1e-4488-99ad-de596999a965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:52.583464 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a54c3e-2c1e-4488-99ad-de596999a965 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:52.587976 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a55e340-357f-4d6e-8a2c-7a284dee2baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:52.588640 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55e340-357f-4d6e-8a2c-7a284dee2baa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:52.589100 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55e340-357f-4d6e-8a2c-7a284dee2baa admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:52.589383 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55e340-357f-4d6e-8a2c-7a284dee2baa admin admin] PATH_INFO: `/v3/users/2f212544d92e4380972f25fa83598d23` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:52.590505 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a55e340-357f-4d6e-8a2c-7a284dee2baa admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f212544d92e4380972f25fa83598d23)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:52.592448 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a55e340-357f-4d6e-8a2c-7a284dee2baa admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:52.766758 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:52.828687 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-3a55e340-357f-4d6e-8a2c-7a284dee2baa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f212544d92e4380972f25fa83598d23'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:52.831787 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-3a55e340-357f-4d6e-8a2c-7a284dee2baa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f212544d92e4380972f25fa83598d23'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:52.841235 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-3a55e340-357f-4d6e-8a2c-7a284dee2baa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f212544d92e4380972f25fa83598d23'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:52.962690 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-3a55e340-357f-4d6e-8a2c-7a284dee2baa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f212544d92e4380972f25fa83598d23'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:52.963410 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-3a55e340-357f-4d6e-8a2c-7a284dee2baa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f212544d92e4380972f25fa83598d23'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:52.985358 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1219/2442] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:08:52 2022] DELETE /identity/v3/users/2f212544d92e4380972f25fa83598d23 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:52.991096 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a30f05-e884-4a1f-9067-b9e10e8e212a None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:52.992076 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a30f05-e884-4a1f-9067-b9e10e8e212a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:52.992625 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a30f05-e884-4a1f-9067-b9e10e8e212a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:53.001199 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:53.019434 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-02a30f05-e884-4a1f-9067-b9e10e8e212a None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:53.021618 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-02a30f05-e884-4a1f-9067-b9e10e8e212a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:53.094089 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1224/2443] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:53.104806 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a55e340-357f-4d6e-8a2c-7a284dee2baa admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:53.106361 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3a55e340-357f-4d6e-8a2c-7a284dee2baa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:53.174872 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a55e340-357f-4d6e-8a2c-7a284dee2baa admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:53.179203 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7643da2-2c4b-422d-8bf3-ab7a2fcef8b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:53.180086 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7643da2-2c4b-422d-8bf3-ab7a2fcef8b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:53.180086 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7643da2-2c4b-422d-8bf3-ab7a2fcef8b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:53.180264 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7643da2-2c4b-422d-8bf3-ab7a2fcef8b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:53.191535 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7643da2-2c4b-422d-8bf3-ab7a2fcef8b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:53.194277 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7643da2-2c4b-422d-8bf3-ab7a2fcef8b7 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:53.207179 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1220/2444] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:08:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 01:08:53.809048 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a30f05-e884-4a1f-9067-b9e10e8e212a None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:53.810738 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-02a30f05-e884-4a1f-9067-b9e10e8e212a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:53.825612 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a30f05-e884-4a1f-9067-b9e10e8e212a None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:53.833776 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a3190-416d-46af-9baa-f469c8bed01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:53.834756 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a3190-416d-46af-9baa-f469c8bed01c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:53.835273 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a3190-416d-46af-9baa-f469c8bed01c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:53.835754 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a3190-416d-46af-9baa-f469c8bed01c admin admin] PATH_INFO: `/v3/projects/1678777938d84d40a794fa79aafc5e42` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:53.844108 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a3190-416d-46af-9baa-f469c8bed01c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1678777938d84d40a794fa79aafc5e42)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:53.844108 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a3190-416d-46af-9baa-f469c8bed01c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:54.011285 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-057a3190-416d-46af-9baa-f469c8bed01c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1678777938d84d40a794fa79aafc5e42', 'request_id': 'req-057a3190-416d-46af-9baa-f469c8bed01c'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:54.147024 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-057a3190-416d-46af-9baa-f469c8bed01c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1678777938d84d40a794fa79aafc5e42', 'request_id': 'req-057a3190-416d-46af-9baa-f469c8bed01c'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:54.161450 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-057a3190-416d-46af-9baa-f469c8bed01c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1678777938d84d40a794fa79aafc5e42', 'request_id': 'req-057a3190-416d-46af-9baa-f469c8bed01c'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:54.183629 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-057a3190-416d-46af-9baa-f469c8bed01c admin admin] The token cache is being invalidate because project 1678777938d84d40a794fa79aafc5e42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:08:54.184019 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-057a3190-416d-46af-9baa-f469c8bed01c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:54.185418 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-057a3190-416d-46af-9baa-f469c8bed01c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:54.187533 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1225/2445] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:08:53 2022] DELETE /identity/v3/projects/1678777938d84d40a794fa79aafc5e42 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:54.191518 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7643da2-2c4b-422d-8bf3-ab7a2fcef8b7 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:54.204550 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7643da2-2c4b-422d-8bf3-ab7a2fcef8b7 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:54.211592 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:54.298048 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914fbcc8-737b-4bd8-94e1-97ae97d01ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:54.298899 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fbcc8-737b-4bd8-94e1-97ae97d01ff5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:54.299233 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fbcc8-737b-4bd8-94e1-97ae97d01ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:54.299521 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fbcc8-737b-4bd8-94e1-97ae97d01ff5 admin admin] PATH_INFO: `/v3/users/01cd27f3969e41eeb0a727e19dc862db` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:54.301770 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914fbcc8-737b-4bd8-94e1-97ae97d01ff5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01cd27f3969e41eeb0a727e19dc862db)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:54.304307 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914fbcc8-737b-4bd8-94e1-97ae97d01ff5 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:54.552998 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-914fbcc8-737b-4bd8-94e1-97ae97d01ff5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01cd27f3969e41eeb0a727e19dc862db'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:54.554180 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-914fbcc8-737b-4bd8-94e1-97ae97d01ff5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01cd27f3969e41eeb0a727e19dc862db'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:54.561629 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-914fbcc8-737b-4bd8-94e1-97ae97d01ff5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01cd27f3969e41eeb0a727e19dc862db'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:54.618139 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-914fbcc8-737b-4bd8-94e1-97ae97d01ff5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01cd27f3969e41eeb0a727e19dc862db'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:54.618906 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-914fbcc8-737b-4bd8-94e1-97ae97d01ff5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01cd27f3969e41eeb0a727e19dc862db'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:54.639464 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1221/2446] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:08:54 2022] DELETE /identity/v3/users/01cd27f3969e41eeb0a727e19dc862db => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:55.242143 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a3190-416d-46af-9baa-f469c8bed01c admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:55.244620 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-057a3190-416d-46af-9baa-f469c8bed01c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:55.283688 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a3190-416d-46af-9baa-f469c8bed01c admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:55.378587 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1273cfc-5eae-4088-b2df-a689cf33eb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:55.379549 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1273cfc-5eae-4088-b2df-a689cf33eb65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:55.379970 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1273cfc-5eae-4088-b2df-a689cf33eb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:55.380865 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1273cfc-5eae-4088-b2df-a689cf33eb65 admin admin] PATH_INFO: `/v3/projects/727112b5c8c9409db00635a6d9d8639b` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:55.395371 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1273cfc-5eae-4088-b2df-a689cf33eb65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=727112b5c8c9409db00635a6d9d8639b)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:55.395371 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1273cfc-5eae-4088-b2df-a689cf33eb65 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:55.532850 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:55.549558 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a1273cfc-5eae-4088-b2df-a689cf33eb65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '727112b5c8c9409db00635a6d9d8639b', 'request_id': 'req-a1273cfc-5eae-4088-b2df-a689cf33eb65'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:55.637827 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a1273cfc-5eae-4088-b2df-a689cf33eb65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '727112b5c8c9409db00635a6d9d8639b', 'request_id': 'req-a1273cfc-5eae-4088-b2df-a689cf33eb65'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:55.657544 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a1273cfc-5eae-4088-b2df-a689cf33eb65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '727112b5c8c9409db00635a6d9d8639b', 'request_id': 'req-a1273cfc-5eae-4088-b2df-a689cf33eb65'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:55.714788 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a1273cfc-5eae-4088-b2df-a689cf33eb65 admin admin] The token cache is being invalidate because project 727112b5c8c9409db00635a6d9d8639b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:08:55.715080 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a1273cfc-5eae-4088-b2df-a689cf33eb65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:55.718906 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a1273cfc-5eae-4088-b2df-a689cf33eb65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:55.727984 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1226/2447] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:08:55 2022] DELETE /identity/v3/projects/727112b5c8c9409db00635a6d9d8639b => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:55.729294 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914fbcc8-737b-4bd8-94e1-97ae97d01ff5 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:55.739570 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-914fbcc8-737b-4bd8-94e1-97ae97d01ff5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:55.757630 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914fbcc8-737b-4bd8-94e1-97ae97d01ff5 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:55.882479 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c6a995-56f9-43fa-8932-7bf793ec73da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:55.883251 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6a995-56f9-43fa-8932-7bf793ec73da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:55.883672 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6a995-56f9-43fa-8932-7bf793ec73da admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:55.884064 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6a995-56f9-43fa-8932-7bf793ec73da admin admin] PATH_INFO: `/v3/users/b680f1fafdcb4f71b14a3aedba84f931` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:55.885454 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c6a995-56f9-43fa-8932-7bf793ec73da admin admin] RBAC: Authorizing `identity:delete_user(user_id=b680f1fafdcb4f71b14a3aedba84f931)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:55.888558 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c6a995-56f9-43fa-8932-7bf793ec73da admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:56.121716 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b4c6a995-56f9-43fa-8932-7bf793ec73da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b680f1fafdcb4f71b14a3aedba84f931'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:56.124028 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b4c6a995-56f9-43fa-8932-7bf793ec73da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b680f1fafdcb4f71b14a3aedba84f931'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:56.139223 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b4c6a995-56f9-43fa-8932-7bf793ec73da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b680f1fafdcb4f71b14a3aedba84f931'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:56.212351 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b4c6a995-56f9-43fa-8932-7bf793ec73da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b680f1fafdcb4f71b14a3aedba84f931'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:56.213356 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b4c6a995-56f9-43fa-8932-7bf793ec73da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b680f1fafdcb4f71b14a3aedba84f931'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:56.246691 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1222/2448] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:08:55 2022] DELETE /identity/v3/users/b680f1fafdcb4f71b14a3aedba84f931 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:08:56.528398 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:56.831481 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1273cfc-5eae-4088-b2df-a689cf33eb65 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:56.834458 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a1273cfc-5eae-4088-b2df-a689cf33eb65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:56.866265 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1273cfc-5eae-4088-b2df-a689cf33eb65 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:56.970351 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce2090-6b7d-4db0-a5d0-350e9532464f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:56.970998 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce2090-6b7d-4db0-a5d0-350e9532464f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:56.971743 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce2090-6b7d-4db0-a5d0-350e9532464f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:56.971743 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce2090-6b7d-4db0-a5d0-350e9532464f admin admin] PATH_INFO: `/v3/projects/fca9d6eccf08433f88ef6d5939a62a82` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:56.973357 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce2090-6b7d-4db0-a5d0-350e9532464f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fca9d6eccf08433f88ef6d5939a62a82)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:56.975959 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce2090-6b7d-4db0-a5d0-350e9532464f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:57.116761 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e3ce2090-6b7d-4db0-a5d0-350e9532464f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fca9d6eccf08433f88ef6d5939a62a82', 'request_id': 'req-e3ce2090-6b7d-4db0-a5d0-350e9532464f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:57.198366 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e3ce2090-6b7d-4db0-a5d0-350e9532464f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fca9d6eccf08433f88ef6d5939a62a82', 'request_id': 'req-e3ce2090-6b7d-4db0-a5d0-350e9532464f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:57.221942 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e3ce2090-6b7d-4db0-a5d0-350e9532464f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fca9d6eccf08433f88ef6d5939a62a82', 'request_id': 'req-e3ce2090-6b7d-4db0-a5d0-350e9532464f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:57.252980 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e3ce2090-6b7d-4db0-a5d0-350e9532464f admin admin] The token cache is being invalidate because project fca9d6eccf08433f88ef6d5939a62a82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:08:57.253551 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e3ce2090-6b7d-4db0-a5d0-350e9532464f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:57.254649 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e3ce2090-6b7d-4db0-a5d0-350e9532464f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:57.256750 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1227/2449] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:08:56 2022] DELETE /identity/v3/projects/fca9d6eccf08433f88ef6d5939a62a82 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:57.670208 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b038b4e-fe55-47d3-9e04-ff53453693c5 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:57.670654 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b038b4e-fe55-47d3-9e04-ff53453693c5 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:57.670857 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b038b4e-fe55-47d3-9e04-ff53453693c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:57.729187 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-9b038b4e-fe55-47d3-9e04-ff53453693c5 None None] MFA Rules not processed for user `7fee1d5bc3da421fa1c85ea59223fb54`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:57.730621 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9b038b4e-fe55-47d3-9e04-ff53453693c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:57.824991 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1223/2450] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:57.829619 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce2090-6b7d-4db0-a5d0-350e9532464f admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:57.831522 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e3ce2090-6b7d-4db0-a5d0-350e9532464f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:57.912672 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce2090-6b7d-4db0-a5d0-350e9532464f admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:57.918383 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8602d60-a936-4e5b-b133-05b3ada09a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:57.919268 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8602d60-a936-4e5b-b133-05b3ada09a86 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:57.919763 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8602d60-a936-4e5b-b133-05b3ada09a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:57.920286 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8602d60-a936-4e5b-b133-05b3ada09a86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:57.921099 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8602d60-a936-4e5b-b133-05b3ada09a86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:57.925233 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8602d60-a936-4e5b-b133-05b3ada09a86 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:58.031650 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b038b4e-fe55-47d3-9e04-ff53453693c5 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:58.031797 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1228/2451] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:08:57 2022] POST /identity/v3/projects => generated 373 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:08:58.051308 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b038b4e-fe55-47d3-9e04-ff53453693c5 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:58.116106 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d125c74-0ec2-46f8-a361-461fb4a03bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:58.116665 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d125c74-0ec2-46f8-a361-461fb4a03bff admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:58.117000 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d125c74-0ec2-46f8-a361-461fb4a03bff admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:58.117262 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d125c74-0ec2-46f8-a361-461fb4a03bff admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:58.117963 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d125c74-0ec2-46f8-a361-461fb4a03bff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:58.119954 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d125c74-0ec2-46f8-a361-461fb4a03bff admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:58.236379 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:58.241770 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1224/2452] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:08:58 2022] POST /identity/v3/users => generated 293 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:08:58.246185 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8602d60-a936-4e5b-b133-05b3ada09a86 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:58.259681 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8602d60-a936-4e5b-b133-05b3ada09a86 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:58.265716 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d03fa-b03a-4a94-aa9c-780bc55ddb6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:58.266362 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d03fa-b03a-4a94-aa9c-780bc55ddb6f admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:58.266649 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d03fa-b03a-4a94-aa9c-780bc55ddb6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:58.266929 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d03fa-b03a-4a94-aa9c-780bc55ddb6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:58.267590 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861d03fa-b03a-4a94-aa9c-780bc55ddb6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:58.269402 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861d03fa-b03a-4a94-aa9c-780bc55ddb6f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:58.284114 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1229/2453] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:08:58.288493 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d125c74-0ec2-46f8-a361-461fb4a03bff admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:58.305733 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d125c74-0ec2-46f8-a361-461fb4a03bff admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:58.310951 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d086b1-c204-4236-b8f2-13371333e906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:58.311659 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d086b1-c204-4236-b8f2-13371333e906 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:58.311948 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d086b1-c204-4236-b8f2-13371333e906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:58.312261 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d086b1-c204-4236-b8f2-13371333e906 admin admin] PATH_INFO: `/v3/projects/07adc8f73fe240fd87ad0c8523874965/users/8bea0cb76e174890939d3558681259f4/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:58.333183 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d086b1-c204-4236-b8f2-13371333e906 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07adc8f73fe240fd87ad0c8523874965, user_id=8bea0cb76e174890939d3558681259f4, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:58.335998 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d086b1-c204-4236-b8f2-13371333e906 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:58.397879 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1225/2454] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:08:58 2022] PUT /identity/v3/projects/07adc8f73fe240fd87ad0c8523874965/users/8bea0cb76e174890939d3558681259f4/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:08:58.401706 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d03fa-b03a-4a94-aa9c-780bc55ddb6f admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:58.412774 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d03fa-b03a-4a94-aa9c-780bc55ddb6f admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:58.476763 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8fcb88-2e9c-4765-a9cd-9b4986f17663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:58.477468 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8fcb88-2e9c-4765-a9cd-9b4986f17663 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:58.477756 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8fcb88-2e9c-4765-a9cd-9b4986f17663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:58.478029 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8fcb88-2e9c-4765-a9cd-9b4986f17663 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:58.481047 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8fcb88-2e9c-4765-a9cd-9b4986f17663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:58.481047 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8fcb88-2e9c-4765-a9cd-9b4986f17663 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:58.494748 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1230/2455] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:08:58.495908 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d086b1-c204-4236-b8f2-13371333e906 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:58.507663 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d086b1-c204-4236-b8f2-13371333e906 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:58.512941 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce498bce-eda5-4934-a269-7ca0e8e7d82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:58.513631 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce498bce-eda5-4934-a269-7ca0e8e7d82e admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:58.513913 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce498bce-eda5-4934-a269-7ca0e8e7d82e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:58.514185 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce498bce-eda5-4934-a269-7ca0e8e7d82e admin admin] PATH_INFO: `/v3/projects/07adc8f73fe240fd87ad0c8523874965/users/8bea0cb76e174890939d3558681259f4/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:58.516177 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce498bce-eda5-4934-a269-7ca0e8e7d82e admin admin] RBAC: Authorizing `identity:create_grant(project_id=07adc8f73fe240fd87ad0c8523874965, user_id=8bea0cb76e174890939d3558681259f4, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:58.518846 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce498bce-eda5-4934-a269-7ca0e8e7d82e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:58.532892 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1226/2456] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:08:58 2022] PUT /identity/v3/projects/07adc8f73fe240fd87ad0c8523874965/users/8bea0cb76e174890939d3558681259f4/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:58.617880 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:59.010674 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8fcb88-2e9c-4765-a9cd-9b4986f17663 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:59.012945 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8c8fcb88-2e9c-4765-a9cd-9b4986f17663 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:59.027330 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8fcb88-2e9c-4765-a9cd-9b4986f17663 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:59.104404 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0902b14-3059-4281-858e-37176a0b6a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:59.105535 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0902b14-3059-4281-858e-37176a0b6a48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:59.106067 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0902b14-3059-4281-858e-37176a0b6a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:59.106591 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0902b14-3059-4281-858e-37176a0b6a48 admin admin] PATH_INFO: `/v3/users/9e1177488c17428eb8286eea3a2b216e` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:59.108414 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0902b14-3059-4281-858e-37176a0b6a48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e1177488c17428eb8286eea3a2b216e)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:59.111369 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0902b14-3059-4281-858e-37176a0b6a48 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:59.393834 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a0902b14-3059-4281-858e-37176a0b6a48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e1177488c17428eb8286eea3a2b216e'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:59.395040 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a0902b14-3059-4281-858e-37176a0b6a48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e1177488c17428eb8286eea3a2b216e'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:59.528773 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a0902b14-3059-4281-858e-37176a0b6a48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e1177488c17428eb8286eea3a2b216e'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:59.530713 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a0902b14-3059-4281-858e-37176a0b6a48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e1177488c17428eb8286eea3a2b216e'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:59.537825 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:59.552655 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a0902b14-3059-4281-858e-37176a0b6a48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e1177488c17428eb8286eea3a2b216e'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:08:59.571988 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1231/2457] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:08:59 2022] DELETE /identity/v3/users/9e1177488c17428eb8286eea3a2b216e => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:08:59.573697 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fa34e-f70a-4424-8a7f-6fcc76484bd3 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:59.574184 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fa34e-f70a-4424-8a7f-6fcc76484bd3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:59.574267 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fa34e-f70a-4424-8a7f-6fcc76484bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:59.596040 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c56fa34e-f70a-4424-8a7f-6fcc76484bd3 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:08:59.597314 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c56fa34e-f70a-4424-8a7f-6fcc76484bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:59.679600 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1227/2458] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:08:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:08:59.690042 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0902b14-3059-4281-858e-37176a0b6a48 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:08:59.691685 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a0902b14-3059-4281-858e-37176a0b6a48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:08:59.752686 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0902b14-3059-4281-858e-37176a0b6a48 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:08:59.758007 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b45eb4-3c52-4394-9c4d-439d8c198e4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:08:59.758725 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b45eb4-3c52-4394-9c4d-439d8c198e4a service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:08:59.759006 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b45eb4-3c52-4394-9c4d-439d8c198e4a service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:08:59.759279 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b45eb4-3c52-4394-9c4d-439d8c198e4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:08:59.769118 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b45eb4-3c52-4394-9c4d-439d8c198e4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:08:59.771398 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b45eb4-3c52-4394-9c4d-439d8c198e4a service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:08:59.782449 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1232/2459] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:08:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 01:09:00.622531 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56fa34e-f70a-4424-8a7f-6fcc76484bd3 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:09:00.626292 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c56fa34e-f70a-4424-8a7f-6fcc76484bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:09:00.639179 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56fa34e-f70a-4424-8a7f-6fcc76484bd3 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:09:00.644180 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56913ec4-c170-4f3d-9297-6c4f2cf19448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:09:00.645046 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56913ec4-c170-4f3d-9297-6c4f2cf19448 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:09:00.645321 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56913ec4-c170-4f3d-9297-6c4f2cf19448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:09:00.647080 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56913ec4-c170-4f3d-9297-6c4f2cf19448 admin admin] PATH_INFO: `/v3/projects/4715bba986c643c285f8cdf105d44a23` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:09:00.647182 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56913ec4-c170-4f3d-9297-6c4f2cf19448 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4715bba986c643c285f8cdf105d44a23)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:09:00.649298 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56913ec4-c170-4f3d-9297-6c4f2cf19448 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:09:00.780169 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-56913ec4-c170-4f3d-9297-6c4f2cf19448 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4715bba986c643c285f8cdf105d44a23', 'request_id': 'req-56913ec4-c170-4f3d-9297-6c4f2cf19448'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:09:00.860258 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-56913ec4-c170-4f3d-9297-6c4f2cf19448 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4715bba986c643c285f8cdf105d44a23', 'request_id': 'req-56913ec4-c170-4f3d-9297-6c4f2cf19448'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:09:00.871562 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-56913ec4-c170-4f3d-9297-6c4f2cf19448 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4715bba986c643c285f8cdf105d44a23', 'request_id': 'req-56913ec4-c170-4f3d-9297-6c4f2cf19448'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:09:00.895062 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-56913ec4-c170-4f3d-9297-6c4f2cf19448 admin admin] The token cache is being invalidate because project 4715bba986c643c285f8cdf105d44a23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:09:00.909472 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-56913ec4-c170-4f3d-9297-6c4f2cf19448 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:09:00.909472 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-56913ec4-c170-4f3d-9297-6c4f2cf19448 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:09:00.909472 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1228/2460] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:09:00 2022] DELETE /identity/v3/projects/4715bba986c643c285f8cdf105d44a23 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:09:05.890745 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5b3f6-36df-474e-8d56-aab0f75b7abc None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:09:05.891239 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5b3f6-36df-474e-8d56-aab0f75b7abc None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:09:05.891239 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5b3f6-36df-474e-8d56-aab0f75b7abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:09:05.919936 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-bed5b3f6-36df-474e-8d56-aab0f75b7abc None None] MFA Rules not processed for user `8bea0cb76e174890939d3558681259f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:09:05.921182 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-bed5b3f6-36df-474e-8d56-aab0f75b7abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:09:05.992372 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1233/2461] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:09:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:09:06.006429 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56913ec4-c170-4f3d-9297-6c4f2cf19448 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:09:06.007872 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-56913ec4-c170-4f3d-9297-6c4f2cf19448 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:09:06.080512 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56913ec4-c170-4f3d-9297-6c4f2cf19448 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:09:06.086435 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9dfc7-b4c1-4aef-aa66-9ddc75e991fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:09:06.087037 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9dfc7-b4c1-4aef-aa66-9ddc75e991fa service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:09:06.087322 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9dfc7-b4c1-4aef-aa66-9ddc75e991fa service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:09:06.087620 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9dfc7-b4c1-4aef-aa66-9ddc75e991fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:09:06.089337 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9ad9dfc7-b4c1-4aef-aa66-9ddc75e991fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:09:06.098974 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:09:06.147311 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad9dfc7-b4c1-4aef-aa66-9ddc75e991fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:09:06.148917 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad9dfc7-b4c1-4aef-aa66-9ddc75e991fa service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:09:06.157798 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1229/2462] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:09:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:09:08.769602 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25f25b-dcd8-49e6-9922-390f248c7111 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:09:08.770124 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25f25b-dcd8-49e6-9922-390f248c7111 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:09:08.770124 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25f25b-dcd8-49e6-9922-390f248c7111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:09:08.791284 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-8f25f25b-dcd8-49e6-9922-390f248c7111 None None] MFA Rules not processed for user `7fee1d5bc3da421fa1c85ea59223fb54`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:09:08.792737 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8f25f25b-dcd8-49e6-9922-390f248c7111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:09:08.799763 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1234/2463] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:09:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:09:16.247916 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=23 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:09:18.784869 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:DELETE=2,keystone:SELECT=72 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:09:28.197875 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17b5f0-eb8a-4566-b014-701c4210a5c3 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:09:28.199979 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17b5f0-eb8a-4566-b014-701c4210a5c3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:09:28.199979 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17b5f0-eb8a-4566-b014-701c4210a5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:09:28.220487 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-7c17b5f0-eb8a-4566-b014-701c4210a5c3 None None] MFA Rules not processed for user `0c5e9731f6204b44841b8524762d5a77`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:09:28.223340 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7c17b5f0-eb8a-4566-b014-701c4210a5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:09:28.287594 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1230/2464] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:09:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:09:38.279639 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=25 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:09:42.634618 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f25f25b-dcd8-49e6-9922-390f248c7111 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:09:42.636314 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8f25f25b-dcd8-49e6-9922-390f248c7111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:09:42.707694 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f25f25b-dcd8-49e6-9922-390f248c7111 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:09:42.712720 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0bb257-bb30-405a-9440-4f46145f626b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '018802a8d9c041119ad708430f885a89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:09:42.713325 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0bb257-bb30-405a-9440-4f46145f626b service placement] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:09:42.713613 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0bb257-bb30-405a-9440-4f46145f626b service placement] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:09:42.713973 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0bb257-bb30-405a-9440-4f46145f626b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:09:42.715822 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2f0bb257-bb30-405a-9440-4f46145f626b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:09:42.727114 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0bb257-bb30-405a-9440-4f46145f626b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:09:42.728746 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0bb257-bb30-405a-9440-4f46145f626b service placement] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:09:42.739729 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1235/2465] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:09:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:09:51.322579 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf4ec5-7d9c-4973-969e-562cf9b96d6a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:09:51.323864 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf4ec5-7d9c-4973-969e-562cf9b96d6a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:09:51.323864 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf4ec5-7d9c-4973-969e-562cf9b96d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:09:51.344089 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-75cf4ec5-7d9c-4973-969e-562cf9b96d6a None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:09:51.345320 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-75cf4ec5-7d9c-4973-969e-562cf9b96d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:09:51.406198 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1231/2466] 10.222.0.55 () {58 vars in 1016 bytes} [Sun Sep 4 01:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:09:51.428392 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0bb257-bb30-405a-9440-4f46145f626b service placement] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:09:51.429868 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2f0bb257-bb30-405a-9440-4f46145f626b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:09:51.445608 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0bb257-bb30-405a-9440-4f46145f626b service placement] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:09:51.452373 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace4bec2-356e-456e-b46d-8370c339b706 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:09:51.452944 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace4bec2-356e-456e-b46d-8370c339b706 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:09:51.453227 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace4bec2-356e-456e-b46d-8370c339b706 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:09:51.453503 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace4bec2-356e-456e-b46d-8370c339b706 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:09:51.461644 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace4bec2-356e-456e-b46d-8370c339b706 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:09:51.463848 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace4bec2-356e-456e-b46d-8370c339b706 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:09:51.473265 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1236/2467] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:09:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:09:54.915165 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921348f4-54ef-4698-8ecd-c2bc3c9432ab None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:09:54.915649 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921348f4-54ef-4698-8ecd-c2bc3c9432ab None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:09:54.915754 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921348f4-54ef-4698-8ecd-c2bc3c9432ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:09:54.935294 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-921348f4-54ef-4698-8ecd-c2bc3c9432ab None None] MFA Rules not processed for user `d6d01ca673a1404c9b4cea51036c595c`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:09:54.936523 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-921348f4-54ef-4698-8ecd-c2bc3c9432ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:09:54.991524 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1232/2468] 10.222.0.27 () {58 vars in 1009 bytes} [Sun Sep 4 01:09:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:09:55.036399 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace4bec2-356e-456e-b46d-8370c339b706 service nova] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:09:55.038050 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ace4bec2-356e-456e-b46d-8370c339b706 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:09:55.114618 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace4bec2-356e-456e-b46d-8370c339b706 service nova] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:09:55.120246 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c862a-f994-4b6a-aa75-527f536568ba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2297d8b3fe8e47eb9fd39a6e3eea8f54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:09:55.120918 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c862a-f994-4b6a-aa75-527f536568ba service swift] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:09:55.121202 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c862a-f994-4b6a-aa75-527f536568ba service swift] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:09:55.121479 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c862a-f994-4b6a-aa75-527f536568ba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:09:55.130680 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949c862a-f994-4b6a-aa75-527f536568ba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:09:55.132308 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949c862a-f994-4b6a-aa75-527f536568ba service swift] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:09:55.142058 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1237/2469] 10.222.0.27 () {60 vars in 1384 bytes} [Sun Sep 4 01:09:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 04 01:09:59.873819 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df417e9-0a86-453e-b205-9a20254ab56e None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:09:59.874799 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df417e9-0a86-453e-b205-9a20254ab56e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:09:59.875081 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df417e9-0a86-453e-b205-9a20254ab56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:09:59.902015 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-8df417e9-0a86-453e-b205-9a20254ab56e None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:09:59.903356 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8df417e9-0a86-453e-b205-9a20254ab56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:09:59.913453 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1233/2470] 10.222.0.55 () {58 vars in 1016 bytes} [Sun Sep 4 01:09:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:09:59.952334 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c862a-f994-4b6a-aa75-527f536568ba service swift] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:09:59.977239 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c862a-f994-4b6a-aa75-527f536568ba service swift] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:09:59.985272 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaefdf85-3340-4c11-9d80-e8db008b4116 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:09:59.986199 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefdf85-3340-4c11-9d80-e8db008b4116 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:09:59.988337 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefdf85-3340-4c11-9d80-e8db008b4116 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:09:59.989215 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefdf85-3340-4c11-9d80-e8db008b4116 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:10:00.002067 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaefdf85-3340-4c11-9d80-e8db008b4116 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:10:00.003681 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaefdf85-3340-4c11-9d80-e8db008b4116 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:10:00.022607 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1238/2471] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:09:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:10:01.671456 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df417e9-0a86-453e-b205-9a20254ab56e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:10:01.673747 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8df417e9-0a86-453e-b205-9a20254ab56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:10:01.741844 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df417e9-0a86-453e-b205-9a20254ab56e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:10:01.745501 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed401f6a-6ae6-4279-9178-19da7e1a16f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:10:01.746109 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed401f6a-6ae6-4279-9178-19da7e1a16f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:10:01.746391 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed401f6a-6ae6-4279-9178-19da7e1a16f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:10:01.746672 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed401f6a-6ae6-4279-9178-19da7e1a16f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:10:01.757966 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed401f6a-6ae6-4279-9178-19da7e1a16f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:10:01.760007 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed401f6a-6ae6-4279-9178-19da7e1a16f4 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:10:01.771703 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1234/2472] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:10:01 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:10:06.110421 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c5a07-8bc2-42f8-88ab-28aacc163187 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:10:06.110977 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c5a07-8bc2-42f8-88ab-28aacc163187 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:10:06.110977 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c5a07-8bc2-42f8-88ab-28aacc163187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:10:06.153171 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-998c5a07-8bc2-42f8-88ab-28aacc163187 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:10:06.154456 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-998c5a07-8bc2-42f8-88ab-28aacc163187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:10:06.163068 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1239/2473] 10.222.0.55 () {58 vars in 1016 bytes} [Sun Sep 4 01:10:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:10:06.187782 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed401f6a-6ae6-4279-9178-19da7e1a16f4 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:10:06.201533 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed401f6a-6ae6-4279-9178-19da7e1a16f4 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:10:06.209466 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d52aec-eda9-443c-8c35-68db873a0801 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:10:06.214472 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d52aec-eda9-443c-8c35-68db873a0801 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:10:06.214472 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d52aec-eda9-443c-8c35-68db873a0801 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:10:06.214472 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d52aec-eda9-443c-8c35-68db873a0801 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:10:06.234241 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d52aec-eda9-443c-8c35-68db873a0801 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:10:06.236380 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d52aec-eda9-443c-8c35-68db873a0801 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:10:06.250561 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1235/2474] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:10:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:10:11.549913 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ffed9-6ae7-4a24-ac47-88a8640b53c0 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:10:11.550411 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ffed9-6ae7-4a24-ac47-88a8640b53c0 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:10:11.550507 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ffed9-6ae7-4a24-ac47-88a8640b53c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:10:11.558434 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=80 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:10:11.570557 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-2d9ffed9-6ae7-4a24-ac47-88a8640b53c0 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:10:11.571906 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2d9ffed9-6ae7-4a24-ac47-88a8640b53c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:10:11.579935 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1240/2475] 10.222.0.55 () {58 vars in 1016 bytes} [Sun Sep 4 01:10:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:10:11.621343 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d52aec-eda9-443c-8c35-68db873a0801 service nova] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:10:11.632452 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d52aec-eda9-443c-8c35-68db873a0801 service nova] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:10:11.638385 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f495456-8fb8-4dd6-8558-db499686f8b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:10:11.638965 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f495456-8fb8-4dd6-8558-db499686f8b9 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:10:11.639243 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f495456-8fb8-4dd6-8558-db499686f8b9 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:10:11.639513 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f495456-8fb8-4dd6-8558-db499686f8b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:10:11.647537 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f495456-8fb8-4dd6-8558-db499686f8b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:10:11.649172 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f495456-8fb8-4dd6-8558-db499686f8b9 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:10:11.659129 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1236/2476] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:10:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:10:16.260642 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c7dbd-590d-43ff-b37d-07ab435157b0 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:10:16.261826 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c7dbd-590d-43ff-b37d-07ab435157b0 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:10:16.261826 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c7dbd-590d-43ff-b37d-07ab435157b0 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:10:16.263046 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1241/2477] 10.222.0.27 () {54 vars in 921 bytes} [Sun Sep 4 01:10:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:10:16.268868 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1225ec-67b3-4e2f-967f-1722f790d1cc None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:10:16.269311 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1225ec-67b3-4e2f-967f-1722f790d1cc None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:10:16.269560 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1225ec-67b3-4e2f-967f-1722f790d1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:10:16.291304 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-8f1225ec-67b3-4e2f-967f-1722f790d1cc None None] MFA Rules not processed for user `68bc00bda5194cd8a7928f8c4df4ccd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:10:16.292717 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8f1225ec-67b3-4e2f-967f-1722f790d1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:10:16.303108 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1237/2478] 10.222.0.27 () {58 vars in 1018 bytes} [Sun Sep 4 01:10:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:10:16.314235 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c7dbd-590d-43ff-b37d-07ab435157b0 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:10:16.316077 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2e1c7dbd-590d-43ff-b37d-07ab435157b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:10:16.376831 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c7dbd-590d-43ff-b37d-07ab435157b0 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:10:16.380347 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc5a6b0-3806-4408-b0f9-098e2d9765fb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1179503bdab547c99175a27b7ce27b9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:10:16.380980 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5a6b0-3806-4408-b0f9-098e2d9765fb service glance] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:10:16.381264 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5a6b0-3806-4408-b0f9-098e2d9765fb service glance] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:10:16.382678 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5a6b0-3806-4408-b0f9-098e2d9765fb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:10:16.392789 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc5a6b0-3806-4408-b0f9-098e2d9765fb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:10:16.394771 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc5a6b0-3806-4408-b0f9-098e2d9765fb service glance] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:10:16.405414 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1242/2479] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:10:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:10:16.556098 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59b7cd-7290-4efc-8289-270ef72b84cb None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:10:16.556379 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59b7cd-7290-4efc-8289-270ef72b84cb None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:10:16.556674 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59b7cd-7290-4efc-8289-270ef72b84cb None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:10:16.558046 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1238/2480] 10.222.0.27 () {54 vars in 921 bytes} [Sun Sep 4 01:10:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:10:16.563008 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec6816-1d85-4865-909c-8c4537ad58d1 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:10:16.563298 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec6816-1d85-4865-909c-8c4537ad58d1 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:10:16.563580 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec6816-1d85-4865-909c-8c4537ad58d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:10:16.584576 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-2cec6816-1d85-4865-909c-8c4537ad58d1 None None] MFA Rules not processed for user `861c278f8bf34b3bb407b2e051119ee9`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:10:16.585802 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2cec6816-1d85-4865-909c-8c4537ad58d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:10:16.593715 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1243/2481] 10.222.0.27 () {58 vars in 1018 bytes} [Sun Sep 4 01:10:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:10:16.605678 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59b7cd-7290-4efc-8289-270ef72b84cb None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:10:16.616425 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59b7cd-7290-4efc-8289-270ef72b84cb None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:10:16.619891 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110b549-9026-4d50-a450-a3c8ca36664d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:10:16.621001 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110b549-9026-4d50-a450-a3c8ca36664d service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:10:16.621277 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110b549-9026-4d50-a450-a3c8ca36664d service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:10:16.621550 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110b549-9026-4d50-a450-a3c8ca36664d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:10:16.626353 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:10:16.628660 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4110b549-9026-4d50-a450-a3c8ca36664d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:10:16.630066 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4110b549-9026-4d50-a450-a3c8ca36664d service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:10:16.638858 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1239/2482] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:10:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:10:16.734683 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44122e8b-8e21-4a3d-986d-79049df786d3 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:10:16.734975 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44122e8b-8e21-4a3d-986d-79049df786d3 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:10:16.735312 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44122e8b-8e21-4a3d-986d-79049df786d3 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:10:16.736861 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1244/2483] 10.222.0.27 () {54 vars in 921 bytes} [Sun Sep 4 01:10:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 04 01:10:16.741629 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b23e93-2904-49f6-bfbe-9bc735c1aaea None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:10:16.741915 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b23e93-2904-49f6-bfbe-9bc735c1aaea None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:10:16.742190 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b23e93-2904-49f6-bfbe-9bc735c1aaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:10:16.763573 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-69b23e93-2904-49f6-bfbe-9bc735c1aaea None None] MFA Rules not processed for user `68bc00bda5194cd8a7928f8c4df4ccd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:10:16.764861 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-69b23e93-2904-49f6-bfbe-9bc735c1aaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:10:16.774316 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1240/2484] 10.222.0.27 () {58 vars in 1018 bytes} [Sun Sep 4 01:10:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:10:16.786143 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44122e8b-8e21-4a3d-986d-79049df786d3 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:10:16.787655 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-44122e8b-8e21-4a3d-986d-79049df786d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:10:16.801834 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44122e8b-8e21-4a3d-986d-79049df786d3 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:10:16.806046 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ffda90-2854-49d6-8ca4-ee282dba544b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:10:16.806650 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffda90-2854-49d6-8ca4-ee282dba544b service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:10:16.806930 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffda90-2854-49d6-8ca4-ee282dba544b service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:10:16.807203 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffda90-2854-49d6-8ca4-ee282dba544b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:10:16.819212 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ffda90-2854-49d6-8ca4-ee282dba544b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:10:16.820677 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ffda90-2854-49d6-8ca4-ee282dba544b service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:10:16.832859 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1245/2485] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:10:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:10:16.857384 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61784ae-763b-4820-855c-c82c51f8cc38 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:10:16.857670 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61784ae-763b-4820-855c-c82c51f8cc38 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:10:16.857954 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61784ae-763b-4820-855c-c82c51f8cc38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:10:16.878667 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c61784ae-763b-4820-855c-c82c51f8cc38 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:10:16.879882 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c61784ae-763b-4820-855c-c82c51f8cc38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:10:16.889342 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1241/2486] 10.222.0.55 () {58 vars in 1016 bytes} [Sun Sep 4 01:10:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:10:16.914010 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ffda90-2854-49d6-8ca4-ee282dba544b service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:10:16.925569 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ffda90-2854-49d6-8ca4-ee282dba544b service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:10:16.931318 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055ded1-6deb-4a90-8cdd-b18c048258e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:10:16.931847 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055ded1-6deb-4a90-8cdd-b18c048258e6 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:10:16.932133 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055ded1-6deb-4a90-8cdd-b18c048258e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:10:16.932411 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055ded1-6deb-4a90-8cdd-b18c048258e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:10:16.941421 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c055ded1-6deb-4a90-8cdd-b18c048258e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:10:16.943007 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c055ded1-6deb-4a90-8cdd-b18c048258e6 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:10:16.953553 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1246/2487] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:10:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:10:21.133399 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26463f3d-4d99-4929-afec-750ad343e331 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:10:21.133842 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26463f3d-4d99-4929-afec-750ad343e331 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:10:21.133959 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26463f3d-4d99-4929-afec-750ad343e331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:10:21.154132 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-26463f3d-4d99-4929-afec-750ad343e331 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:10:21.155370 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-26463f3d-4d99-4929-afec-750ad343e331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:10:21.163469 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1242/2488] 10.222.0.55 () {58 vars in 1016 bytes} [Sun Sep 4 01:10:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:10:21.188756 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055ded1-6deb-4a90-8cdd-b18c048258e6 service nova] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:10:21.201278 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055ded1-6deb-4a90-8cdd-b18c048258e6 service nova] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:10:21.207050 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bfb303-e22f-4524-8b70-d0d3494aed55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:10:21.207655 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bfb303-e22f-4524-8b70-d0d3494aed55 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:10:21.207938 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bfb303-e22f-4524-8b70-d0d3494aed55 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:10:21.208232 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bfb303-e22f-4524-8b70-d0d3494aed55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:10:21.217087 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bfb303-e22f-4524-8b70-d0d3494aed55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:10:21.218697 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bfb303-e22f-4524-8b70-d0d3494aed55 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:10:21.233088 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1247/2489] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:10:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:10:24.206637 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee1d5c-ee0a-4be3-a3cd-20fe602baad0 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:10:24.207094 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee1d5c-ee0a-4be3-a3cd-20fe602baad0 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:10:24.207186 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee1d5c-ee0a-4be3-a3cd-20fe602baad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:10:24.228269 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-cdee1d5c-ee0a-4be3-a3cd-20fe602baad0 None None] MFA Rules not processed for user `d6d01ca673a1404c9b4cea51036c595c`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:10:24.230085 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-cdee1d5c-ee0a-4be3-a3cd-20fe602baad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:10:24.238582 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1243/2490] 10.222.0.27 () {58 vars in 1009 bytes} [Sun Sep 4 01:10:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:10:24.251136 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bfb303-e22f-4524-8b70-d0d3494aed55 service nova] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:10:24.262786 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bfb303-e22f-4524-8b70-d0d3494aed55 service nova] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:10:24.268259 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69ea83-bbfd-4852-af01-94d4498ced6d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2297d8b3fe8e47eb9fd39a6e3eea8f54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:10:24.268444 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69ea83-bbfd-4852-af01-94d4498ced6d service swift] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:10:24.268655 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69ea83-bbfd-4852-af01-94d4498ced6d service swift] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:10:24.268936 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69ea83-bbfd-4852-af01-94d4498ced6d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:10:24.277429 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb69ea83-bbfd-4852-af01-94d4498ced6d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:10:24.279032 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb69ea83-bbfd-4852-af01-94d4498ced6d service swift] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:10:24.287131 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1248/2491] 10.222.0.27 () {60 vars in 1384 bytes} [Sun Sep 4 01:10:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 04 01:10:34.222401 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=14 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:10:34.283707 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=55 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:10:54.292547 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182e2a3-adeb-43a7-8c1e-413a5e2dfd94 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:10:54.293076 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182e2a3-adeb-43a7-8c1e-413a5e2dfd94 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:10:54.293302 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182e2a3-adeb-43a7-8c1e-413a5e2dfd94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:10:54.313845 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f182e2a3-adeb-43a7-8c1e-413a5e2dfd94 None None] MFA Rules not processed for user `d6d01ca673a1404c9b4cea51036c595c`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:10:54.315018 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f182e2a3-adeb-43a7-8c1e-413a5e2dfd94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:10:54.322595 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1244/2492] 10.222.0.27 () {58 vars in 967 bytes} [Sun Sep 4 01:10:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:10:54.335653 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69ea83-bbfd-4852-af01-94d4498ced6d service swift] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:10:54.349353 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69ea83-bbfd-4852-af01-94d4498ced6d service swift] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:10:54.354751 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40cafc9-2f19-4ea1-885d-108fb4319437 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2297d8b3fe8e47eb9fd39a6e3eea8f54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:10:54.355345 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40cafc9-2f19-4ea1-885d-108fb4319437 service swift] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:10:54.355623 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40cafc9-2f19-4ea1-885d-108fb4319437 service swift] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:10:54.355895 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40cafc9-2f19-4ea1-885d-108fb4319437 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:10:54.366149 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40cafc9-2f19-4ea1-885d-108fb4319437 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:10:54.367819 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40cafc9-2f19-4ea1-885d-108fb4319437 service swift] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:10:54.376308 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1249/2493] 10.222.0.27 () {60 vars in 1384 bytes} [Sun Sep 4 01:10:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 04 01:11:00.843904 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4123cd-3bcb-4c93-86c5-249b0cb29562 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:11:00.844375 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4123cd-3bcb-4c93-86c5-249b0cb29562 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:11:00.844503 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4123cd-3bcb-4c93-86c5-249b0cb29562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:11:00.864555 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5c4123cd-3bcb-4c93-86c5-249b0cb29562 None None] MFA Rules not processed for user `6ddda0a1a0774b2da6bc75cbb10e094b`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:11:00.865778 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5c4123cd-3bcb-4c93-86c5-249b0cb29562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:11:00.918808 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1245/2494] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:11:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:11:04.373008 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=6 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:11:10.912323 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=28 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:11:14.098156 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ef9b1-0555-4081-8285-16903089a99b None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:11:14.098660 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ef9b1-0555-4081-8285-16903089a99b None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:11:14.098777 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ef9b1-0555-4081-8285-16903089a99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:11:14.118854 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-1d0ef9b1-0555-4081-8285-16903089a99b None None] MFA Rules not processed for user `9bef8dc1da4c44dbbafcaf3c813b3262`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:11:14.120045 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-1d0ef9b1-0555-4081-8285-16903089a99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:11:14.175756 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1250/2495] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:11:14 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:11:23.848781 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27a97c-af4d-4c1a-9932-667c34d3d059 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:11:23.849332 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27a97c-af4d-4c1a-9932-667c34d3d059 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:11:23.849451 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27a97c-af4d-4c1a-9932-667c34d3d059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:11:23.869024 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-ed27a97c-af4d-4c1a-9932-667c34d3d059 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:11:23.870280 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ed27a97c-af4d-4c1a-9932-667c34d3d059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:11:23.878557 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1246/2496] 10.222.0.55 () {58 vars in 1016 bytes} [Sun Sep 4 01:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:11:23.899567 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ef9b1-0555-4081-8285-16903089a99b None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:11:23.912445 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ef9b1-0555-4081-8285-16903089a99b None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:11:23.917900 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e0d4d0-f6cb-41d8-90a7-b9f2b48a96b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:11:23.918532 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0d4d0-f6cb-41d8-90a7-b9f2b48a96b3 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:11:23.918812 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0d4d0-f6cb-41d8-90a7-b9f2b48a96b3 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:11:23.919087 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0d4d0-f6cb-41d8-90a7-b9f2b48a96b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:11:23.927357 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e0d4d0-f6cb-41d8-90a7-b9f2b48a96b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:11:23.929014 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e0d4d0-f6cb-41d8-90a7-b9f2b48a96b3 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:11:23.938555 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1251/2497] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:11:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:11:28.447412 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05adeb-677e-4b6b-b7c6-27a99b35737f None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:11:28.448004 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05adeb-677e-4b6b-b7c6-27a99b35737f None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:11:28.448004 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05adeb-677e-4b6b-b7c6-27a99b35737f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:11:28.468433 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-df05adeb-677e-4b6b-b7c6-27a99b35737f None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:11:28.469660 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-df05adeb-677e-4b6b-b7c6-27a99b35737f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:11:28.477415 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1247/2498] 10.222.0.55 () {58 vars in 1016 bytes} [Sun Sep 4 01:11:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:11:28.498667 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e0d4d0-f6cb-41d8-90a7-b9f2b48a96b3 service nova] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:11:28.511024 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e0d4d0-f6cb-41d8-90a7-b9f2b48a96b3 service nova] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:11:28.516533 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c5a10-2757-40d2-94ea-5724c8918611 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:11:28.517128 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c5a10-2757-40d2-94ea-5724c8918611 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:11:28.517409 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c5a10-2757-40d2-94ea-5724c8918611 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:11:28.517682 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c5a10-2757-40d2-94ea-5724c8918611 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:11:28.526397 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c5a10-2757-40d2-94ea-5724c8918611 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:11:28.528017 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c5a10-2757-40d2-94ea-5724c8918611 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:11:28.537261 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1252/2499] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:11:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:11:30.928219 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05adeb-677e-4b6b-b7c6-27a99b35737f None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:11:30.929622 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-df05adeb-677e-4b6b-b7c6-27a99b35737f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:11:30.985647 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05adeb-677e-4b6b-b7c6-27a99b35737f None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:11:30.989217 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b39358-b5c2-4a50-b975-fedacd962ec9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:11:30.989812 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b39358-b5c2-4a50-b975-fedacd962ec9 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:11:30.990093 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b39358-b5c2-4a50-b975-fedacd962ec9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:11:30.990855 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b39358-b5c2-4a50-b975-fedacd962ec9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:11:30.992604 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b6b39358-b5c2-4a50-b975-fedacd962ec9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:11:31.005496 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b39358-b5c2-4a50-b975-fedacd962ec9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:11:31.007491 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b39358-b5c2-4a50-b975-fedacd962ec9 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:11:31.018309 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1248/2500] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:11:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:11:32.303522 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc25c45-2f7a-4ea5-a74b-ed0ad76ad1d7 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:11:32.304220 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc25c45-2f7a-4ea5-a74b-ed0ad76ad1d7 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:11:32.304220 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc25c45-2f7a-4ea5-a74b-ed0ad76ad1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:11:32.323485 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-3fc25c45-2f7a-4ea5-a74b-ed0ad76ad1d7 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:11:32.324722 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3fc25c45-2f7a-4ea5-a74b-ed0ad76ad1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:11:32.332578 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1253/2501] 10.222.0.55 () {58 vars in 1016 bytes} [Sun Sep 4 01:11:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:11:32.367724 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b39358-b5c2-4a50-b975-fedacd962ec9 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:11:32.378901 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b39358-b5c2-4a50-b975-fedacd962ec9 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:11:32.385050 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b99e6d-3ffc-4d47-b3f1-395085abda60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:11:32.385639 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b99e6d-3ffc-4d47-b3f1-395085abda60 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:11:32.385915 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b99e6d-3ffc-4d47-b3f1-395085abda60 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:11:32.386187 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b99e6d-3ffc-4d47-b3f1-395085abda60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:11:32.394837 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b99e6d-3ffc-4d47-b3f1-395085abda60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:11:32.396414 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b99e6d-3ffc-4d47-b3f1-395085abda60 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:11:32.406317 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1249/2502] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:11:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:11:33.800082 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc25c45-2f7a-4ea5-a74b-ed0ad76ad1d7 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:11:33.809639 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc25c45-2f7a-4ea5-a74b-ed0ad76ad1d7 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:11:33.813136 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e8038e-d3a6-4fc0-af29-80c6793f0f36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:11:33.813721 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8038e-d3a6-4fc0-af29-80c6793f0f36 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:11:33.813997 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8038e-d3a6-4fc0-af29-80c6793f0f36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:11:33.814267 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8038e-d3a6-4fc0-af29-80c6793f0f36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:11:33.815929 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a5e8038e-d3a6-4fc0-af29-80c6793f0f36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:11:33.829560 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e8038e-d3a6-4fc0-af29-80c6793f0f36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:11:33.830981 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e8038e-d3a6-4fc0-af29-80c6793f0f36 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:11:33.842873 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1254/2503] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:11:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:11:37.584117 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6375b-3dea-4c20-b41c-339ef90e6eeb None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:11:37.585159 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6375b-3dea-4c20-b41c-339ef90e6eeb None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:11:37.585303 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6375b-3dea-4c20-b41c-339ef90e6eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:11:37.606040 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-ece6375b-3dea-4c20-b41c-339ef90e6eeb None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:11:37.607228 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ece6375b-3dea-4c20-b41c-339ef90e6eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:11:37.614953 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1250/2504] 10.222.0.55 () {58 vars in 1016 bytes} [Sun Sep 4 01:11:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:11:37.642141 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e8038e-d3a6-4fc0-af29-80c6793f0f36 service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:11:37.648407 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=47 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:11:37.654336 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e8038e-d3a6-4fc0-af29-80c6793f0f36 service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:11:37.659874 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f0bb7-3c4d-4e24-adfa-428922a5d752 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:11:37.660493 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f0bb7-3c4d-4e24-adfa-428922a5d752 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:11:37.660785 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f0bb7-3c4d-4e24-adfa-428922a5d752 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:11:37.661060 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f0bb7-3c4d-4e24-adfa-428922a5d752 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:11:37.669776 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96f0bb7-3c4d-4e24-adfa-428922a5d752 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:11:37.671386 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96f0bb7-3c4d-4e24-adfa-428922a5d752 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:11:37.681490 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1255/2505] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:11:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:11:42.877490 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8de4ad-de94-4bf4-bc2f-a4fec289b131 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:11:42.878102 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8de4ad-de94-4bf4-bc2f-a4fec289b131 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:11:42.878102 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8de4ad-de94-4bf4-bc2f-a4fec289b131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:11:42.898060 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-8e8de4ad-de94-4bf4-bc2f-a4fec289b131 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:11:42.899277 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8e8de4ad-de94-4bf4-bc2f-a4fec289b131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:11:42.906986 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1251/2506] 10.222.0.55 () {58 vars in 1016 bytes} [Sun Sep 4 01:11:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:11:42.935088 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f0bb7-3c4d-4e24-adfa-428922a5d752 service nova] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:11:42.946124 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f0bb7-3c4d-4e24-adfa-428922a5d752 service nova] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:11:42.951730 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d00090-d680-432b-849b-e888acd18749 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:11:42.952324 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d00090-d680-432b-849b-e888acd18749 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:11:42.952645 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d00090-d680-432b-849b-e888acd18749 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:11:42.952942 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d00090-d680-432b-849b-e888acd18749 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:11:42.961011 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d00090-d680-432b-849b-e888acd18749 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:11:42.962671 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d00090-d680-432b-849b-e888acd18749 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:11:42.972081 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1256/2507] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:11:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:11:44.274960 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8de4ad-de94-4bf4-bc2f-a4fec289b131 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:11:44.287342 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8de4ad-de94-4bf4-bc2f-a4fec289b131 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:11:44.293431 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582306d3-d6fb-443a-a8cf-c932b509e4a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:11:44.294019 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582306d3-d6fb-443a-a8cf-c932b509e4a7 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:11:44.294293 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582306d3-d6fb-443a-a8cf-c932b509e4a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:11:44.294563 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582306d3-d6fb-443a-a8cf-c932b509e4a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:11:44.296233 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-582306d3-d6fb-443a-a8cf-c932b509e4a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:11:44.306485 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582306d3-d6fb-443a-a8cf-c932b509e4a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:11:44.308025 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582306d3-d6fb-443a-a8cf-c932b509e4a7 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:11:44.316731 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1252/2508] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:11:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:11:52.185637 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bead41-2466-4f7c-9236-52c779a1534a None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:11:52.186214 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bead41-2466-4f7c-9236-52c779a1534a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:11:52.186214 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bead41-2466-4f7c-9236-52c779a1534a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:11:52.206749 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-29bead41-2466-4f7c-9236-52c779a1534a None None] MFA Rules not processed for user `0c5e9731f6204b44841b8524762d5a77`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:11:52.208197 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-29bead41-2466-4f7c-9236-52c779a1534a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:11:52.219407 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1257/2509] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:11:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:11:52.578804 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582306d3-d6fb-443a-a8cf-c932b509e4a7 service nova] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:11:52.583318 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=51 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:11:52.588923 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582306d3-d6fb-443a-a8cf-c932b509e4a7 service nova] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:11:52.592982 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e669e75-04f7-4482-9720-8e0429d064fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:11:52.593587 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e669e75-04f7-4482-9720-8e0429d064fe service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:11:52.593865 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e669e75-04f7-4482-9720-8e0429d064fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:11:52.594138 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e669e75-04f7-4482-9720-8e0429d064fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:11:52.596260 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3e669e75-04f7-4482-9720-8e0429d064fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:11:52.606951 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e669e75-04f7-4482-9720-8e0429d064fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:11:52.608438 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e669e75-04f7-4482-9720-8e0429d064fe service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:11:52.617094 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1253/2510] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:11:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:11:53.030126 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bead41-2466-4f7c-9236-52c779a1534a None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:11:53.042532 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bead41-2466-4f7c-9236-52c779a1534a None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:11:53.047577 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc11013-8794-4e51-bd14-8bdab701f048 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '018802a8d9c041119ad708430f885a89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:11:53.048191 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc11013-8794-4e51-bd14-8bdab701f048 service placement] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:11:53.048489 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc11013-8794-4e51-bd14-8bdab701f048 service placement] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:11:53.048790 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc11013-8794-4e51-bd14-8bdab701f048 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:11:53.050553 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-cbc11013-8794-4e51-bd14-8bdab701f048 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:11:53.063801 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc11013-8794-4e51-bd14-8bdab701f048 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:11:53.065446 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc11013-8794-4e51-bd14-8bdab701f048 service placement] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:11:53.077997 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1258/2511] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:11:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:11:59.578829 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e669e75-04f7-4482-9720-8e0429d064fe service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:11:59.587457 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e669e75-04f7-4482-9720-8e0429d064fe service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:11:59.591418 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a5b325-fbb2-44d3-903e-e5067bf2e09d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:11:59.592012 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5b325-fbb2-44d3-903e-e5067bf2e09d service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:11:59.592312 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5b325-fbb2-44d3-903e-e5067bf2e09d service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:11:59.592628 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5b325-fbb2-44d3-903e-e5067bf2e09d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:11:59.595043 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-72a5b325-fbb2-44d3-903e-e5067bf2e09d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:11:59.607005 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a5b325-fbb2-44d3-903e-e5067bf2e09d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:11:59.607968 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a5b325-fbb2-44d3-903e-e5067bf2e09d service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:11:59.616514 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1254/2512] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:11:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:12:03.072722 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=20 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:12:03.806616 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc11013-8794-4e51-bd14-8bdab701f048 service placement] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:03.819438 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc11013-8794-4e51-bd14-8bdab701f048 service placement] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:03.825221 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a06aec5-d965-436a-8572-bb3a70e9a912 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:03.825820 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06aec5-d965-436a-8572-bb3a70e9a912 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:03.837421 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06aec5-d965-436a-8572-bb3a70e9a912 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:03.837421 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06aec5-d965-436a-8572-bb3a70e9a912 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:03.837421 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8a06aec5-d965-436a-8572-bb3a70e9a912 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:12:03.887875 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a06aec5-d965-436a-8572-bb3a70e9a912 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:03.889650 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a06aec5-d965-436a-8572-bb3a70e9a912 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:03.899302 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1259/2513] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:12:03 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:12:09.612752 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=11 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:12:11.400220 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a5b325-fbb2-44d3-903e-e5067bf2e09d service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:11.415807 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a5b325-fbb2-44d3-903e-e5067bf2e09d service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:11.423427 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2d1696-e402-489b-b5bb-1e2a9b08226d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:11.424183 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d1696-e402-489b-b5bb-1e2a9b08226d service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:11.424578 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d1696-e402-489b-b5bb-1e2a9b08226d service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:11.424929 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d1696-e402-489b-b5bb-1e2a9b08226d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:11.426861 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1c2d1696-e402-489b-b5bb-1e2a9b08226d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:12:11.441028 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2d1696-e402-489b-b5bb-1e2a9b08226d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:11.442980 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2d1696-e402-489b-b5bb-1e2a9b08226d service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:11.454980 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1255/2514] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:12:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:12:11.880913 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a06aec5-d965-436a-8572-bb3a70e9a912 service nova] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:11.894630 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a06aec5-d965-436a-8572-bb3a70e9a912 service nova] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:11.900233 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f70aec4-d205-4019-bd1b-f5705bb68d59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:11.900881 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70aec4-d205-4019-bd1b-f5705bb68d59 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:11.901158 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70aec4-d205-4019-bd1b-f5705bb68d59 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:11.901628 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70aec4-d205-4019-bd1b-f5705bb68d59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:11.903538 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5f70aec4-d205-4019-bd1b-f5705bb68d59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:12:11.916728 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f70aec4-d205-4019-bd1b-f5705bb68d59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:11.918594 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f70aec4-d205-4019-bd1b-f5705bb68d59 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:11.930877 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1260/2515] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:12:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:12:16.043005 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd547a7-e779-44f7-adc7-24ce0579a3b3 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:16.043524 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd547a7-e779-44f7-adc7-24ce0579a3b3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:16.043629 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd547a7-e779-44f7-adc7-24ce0579a3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:16.062489 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5cd547a7-e779-44f7-adc7-24ce0579a3b3 None None] MFA Rules not processed for user `bd1c41fcdc724a128117a5288b8559f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:12:16.063678 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5cd547a7-e779-44f7-adc7-24ce0579a3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:12:16.070674 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1256/2516] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:12:16 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:12:21.890159 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f70aec4-d205-4019-bd1b-f5705bb68d59 service nova] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:21.891655 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5f70aec4-d205-4019-bd1b-f5705bb68d59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:12:21.903375 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f70aec4-d205-4019-bd1b-f5705bb68d59 service nova] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:21.907273 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc1c6ac-815f-4c8f-88df-bfc47c303ae1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:21.907990 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1c6ac-815f-4c8f-88df-bfc47c303ae1 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:21.908288 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1c6ac-815f-4c8f-88df-bfc47c303ae1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:21.908586 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1c6ac-815f-4c8f-88df-bfc47c303ae1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:21.910314 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-dbc1c6ac-815f-4c8f-88df-bfc47c303ae1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:12:21.922001 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc1c6ac-815f-4c8f-88df-bfc47c303ae1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:21.923435 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc1c6ac-815f-4c8f-88df-bfc47c303ae1 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:21.932561 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1261/2517] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:12:21 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:12:22.254879 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b53e6-af95-44f7-a462-acf2c5f6bed1 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:22.259683 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b53e6-af95-44f7-a462-acf2c5f6bed1 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:22.259683 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b53e6-af95-44f7-a462-acf2c5f6bed1 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:22.259683 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1257/2518] 10.222.0.27 () {54 vars in 961 bytes} [Sun Sep 4 01:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:12:22.262946 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e71635-80ac-4a28-a650-61fd60a292fb None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:22.263233 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e71635-80ac-4a28-a650-61fd60a292fb None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:22.263509 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e71635-80ac-4a28-a650-61fd60a292fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:22.286769 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-b2e71635-80ac-4a28-a650-61fd60a292fb None None] MFA Rules not processed for user `018802a8d9c041119ad708430f885a89`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:12:22.288219 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b2e71635-80ac-4a28-a650-61fd60a292fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:12:22.299533 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1262/2519] 10.222.0.27 () {58 vars in 1058 bytes} [Sun Sep 4 01:12:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:12:22.310248 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b53e6-af95-44f7-a462-acf2c5f6bed1 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:22.322302 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b53e6-af95-44f7-a462-acf2c5f6bed1 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:22.328272 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca96c34-842b-4c6b-8a0a-c94339ba8910 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '018802a8d9c041119ad708430f885a89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:22.328900 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca96c34-842b-4c6b-8a0a-c94339ba8910 service placement] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:22.329204 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca96c34-842b-4c6b-8a0a-c94339ba8910 service placement] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:22.329472 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca96c34-842b-4c6b-8a0a-c94339ba8910 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:22.331209 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1ca96c34-842b-4c6b-8a0a-c94339ba8910 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:12:22.343396 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca96c34-842b-4c6b-8a0a-c94339ba8910 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:22.345871 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca96c34-842b-4c6b-8a0a-c94339ba8910 service placement] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:22.356168 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1258/2520] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:12:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 01:12:22.358226 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e71635-80ac-4a28-a650-61fd60a292fb None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:22.359706 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b2e71635-80ac-4a28-a650-61fd60a292fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:12:22.375336 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e71635-80ac-4a28-a650-61fd60a292fb None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:22.379692 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547db4ee-fe82-4b43-ab74-af2a53a68cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:22.380401 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547db4ee-fe82-4b43-ab74-af2a53a68cfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:22.380906 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547db4ee-fe82-4b43-ab74-af2a53a68cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:22.381248 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547db4ee-fe82-4b43-ab74-af2a53a68cfb admin admin] PATH_INFO: `/v3/users/6ddda0a1a0774b2da6bc75cbb10e094b` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:22.382523 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547db4ee-fe82-4b43-ab74-af2a53a68cfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ddda0a1a0774b2da6bc75cbb10e094b)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:22.384447 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547db4ee-fe82-4b43-ab74-af2a53a68cfb admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:22.586799 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-547db4ee-fe82-4b43-ab74-af2a53a68cfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ddda0a1a0774b2da6bc75cbb10e094b'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:22.587498 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-547db4ee-fe82-4b43-ab74-af2a53a68cfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ddda0a1a0774b2da6bc75cbb10e094b'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:22.658080 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-547db4ee-fe82-4b43-ab74-af2a53a68cfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ddda0a1a0774b2da6bc75cbb10e094b'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:22.658968 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-547db4ee-fe82-4b43-ab74-af2a53a68cfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ddda0a1a0774b2da6bc75cbb10e094b'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:22.673616 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-547db4ee-fe82-4b43-ab74-af2a53a68cfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ddda0a1a0774b2da6bc75cbb10e094b'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:22.689841 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1263/2521] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:12:22 2022] DELETE /identity/v3/users/6ddda0a1a0774b2da6bc75cbb10e094b => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:12:22.899890 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca96c34-842b-4c6b-8a0a-c94339ba8910 service placement] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:22.912011 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca96c34-842b-4c6b-8a0a-c94339ba8910 service placement] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:22.976737 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b7577-dc22-4fca-a693-5310ee155f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:22.978505 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b7577-dc22-4fca-a693-5310ee155f06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:22.978784 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b7577-dc22-4fca-a693-5310ee155f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:22.979059 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b7577-dc22-4fca-a693-5310ee155f06 admin admin] PATH_INFO: `/v3/projects/18076036cce84d48bc9df44f5b563bac` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:22.980184 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365b7577-dc22-4fca-a693-5310ee155f06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18076036cce84d48bc9df44f5b563bac)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:22.982937 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365b7577-dc22-4fca-a693-5310ee155f06 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:23.081420 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-365b7577-dc22-4fca-a693-5310ee155f06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18076036cce84d48bc9df44f5b563bac', 'request_id': 'req-365b7577-dc22-4fca-a693-5310ee155f06'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:23.166445 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-365b7577-dc22-4fca-a693-5310ee155f06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18076036cce84d48bc9df44f5b563bac', 'request_id': 'req-365b7577-dc22-4fca-a693-5310ee155f06'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:23.172985 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-365b7577-dc22-4fca-a693-5310ee155f06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18076036cce84d48bc9df44f5b563bac', 'request_id': 'req-365b7577-dc22-4fca-a693-5310ee155f06'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:23.184307 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-365b7577-dc22-4fca-a693-5310ee155f06 admin admin] The token cache is being invalidate because project 18076036cce84d48bc9df44f5b563bac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:12:23.184785 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-365b7577-dc22-4fca-a693-5310ee155f06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:23.185456 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-365b7577-dc22-4fca-a693-5310ee155f06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:23.187029 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1259/2522] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:12:22 2022] DELETE /identity/v3/projects/18076036cce84d48bc9df44f5b563bac => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:12:26.413435 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547db4ee-fe82-4b43-ab74-af2a53a68cfb admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:26.413435 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-547db4ee-fe82-4b43-ab74-af2a53a68cfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:12:26.455408 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:12:26.487402 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547db4ee-fe82-4b43-ab74-af2a53a68cfb admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:26.554156 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a1356-9716-48f4-99b3-b771e34016cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:26.554771 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a1356-9716-48f4-99b3-b771e34016cf service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:26.555058 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a1356-9716-48f4-99b3-b771e34016cf service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:26.555338 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a1356-9716-48f4-99b3-b771e34016cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:26.557109 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-243a1356-9716-48f4-99b3-b771e34016cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:12:26.569387 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a1356-9716-48f4-99b3-b771e34016cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:26.571019 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a1356-9716-48f4-99b3-b771e34016cf service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:26.582094 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1264/2523] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:12:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:12:33.181057 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=66,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:12:36.578207 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=45 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:12:46.113526 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b7577-dc22-4fca-a693-5310ee155f06 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:46.115006 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-365b7577-dc22-4fca-a693-5310ee155f06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:12:46.172875 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b7577-dc22-4fca-a693-5310ee155f06 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:46.176443 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d00d2-8026-459f-bf92-95b09d44b7d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:46.177077 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d00d2-8026-459f-bf92-95b09d44b7d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:46.177355 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d00d2-8026-459f-bf92-95b09d44b7d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:46.177664 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d00d2-8026-459f-bf92-95b09d44b7d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:46.179398 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-524d00d2-8026-459f-bf92-95b09d44b7d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:12:46.244872 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d00d2-8026-459f-bf92-95b09d44b7d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:46.246829 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d00d2-8026-459f-bf92-95b09d44b7d6 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:46.257360 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1260/2524] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:12:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:12:49.885753 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a1356-9716-48f4-99b3-b771e34016cf service nova] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:49.887264 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-243a1356-9716-48f4-99b3-b771e34016cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:12:49.963380 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a1356-9716-48f4-99b3-b771e34016cf service nova] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:49.969430 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c57d7-e1b7-4907-8e53-e9ff170e4ef6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '018802a8d9c041119ad708430f885a89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:49.972015 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c57d7-e1b7-4907-8e53-e9ff170e4ef6 service placement] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:49.972015 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c57d7-e1b7-4907-8e53-e9ff170e4ef6 service placement] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:49.972015 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c57d7-e1b7-4907-8e53-e9ff170e4ef6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:49.972413 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-343c57d7-e1b7-4907-8e53-e9ff170e4ef6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:12:49.983883 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343c57d7-e1b7-4907-8e53-e9ff170e4ef6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:49.985509 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343c57d7-e1b7-4907-8e53-e9ff170e4ef6 service placement] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:49.995142 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1265/2525] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:12:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:12:50.059507 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d00d2-8026-459f-bf92-95b09d44b7d6 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:50.063854 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-524d00d2-8026-459f-bf92-95b09d44b7d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:12:50.073498 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d00d2-8026-459f-bf92-95b09d44b7d6 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:50.078797 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189f665-01b1-4cd8-8905-97dbca0c9443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:50.079415 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189f665-01b1-4cd8-8905-97dbca0c9443 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:50.079746 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189f665-01b1-4cd8-8905-97dbca0c9443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:50.080052 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189f665-01b1-4cd8-8905-97dbca0c9443 admin admin] PATH_INFO: `/v3/users/9bef8dc1da4c44dbbafcaf3c813b3262` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:50.081195 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189f665-01b1-4cd8-8905-97dbca0c9443 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bef8dc1da4c44dbbafcaf3c813b3262)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:50.083099 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189f665-01b1-4cd8-8905-97dbca0c9443 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:50.248997 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2189f665-01b1-4cd8-8905-97dbca0c9443 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bef8dc1da4c44dbbafcaf3c813b3262'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:50.249640 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2189f665-01b1-4cd8-8905-97dbca0c9443 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bef8dc1da4c44dbbafcaf3c813b3262'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:50.256361 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2189f665-01b1-4cd8-8905-97dbca0c9443 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bef8dc1da4c44dbbafcaf3c813b3262'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:50.314916 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2189f665-01b1-4cd8-8905-97dbca0c9443 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bef8dc1da4c44dbbafcaf3c813b3262'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:50.315536 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2189f665-01b1-4cd8-8905-97dbca0c9443 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bef8dc1da4c44dbbafcaf3c813b3262'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:50.333758 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1261/2526] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:12:50 2022] DELETE /identity/v3/users/9bef8dc1da4c44dbbafcaf3c813b3262 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:12:50.546033 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c57d7-e1b7-4907-8e53-e9ff170e4ef6 service placement] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:50.557066 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c57d7-e1b7-4907-8e53-e9ff170e4ef6 service placement] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:50.615841 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84ef6b-037a-4469-bc71-a63b06b020c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:50.616489 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84ef6b-037a-4469-bc71-a63b06b020c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:50.616761 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84ef6b-037a-4469-bc71-a63b06b020c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:50.617034 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84ef6b-037a-4469-bc71-a63b06b020c3 admin admin] PATH_INFO: `/v3/projects/51dbf47189fc4f3c90eadc4887a831b8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:50.618136 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc84ef6b-037a-4469-bc71-a63b06b020c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51dbf47189fc4f3c90eadc4887a831b8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:50.620056 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc84ef6b-037a-4469-bc71-a63b06b020c3 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:50.721125 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-cc84ef6b-037a-4469-bc71-a63b06b020c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51dbf47189fc4f3c90eadc4887a831b8', 'request_id': 'req-cc84ef6b-037a-4469-bc71-a63b06b020c3'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:50.789453 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-cc84ef6b-037a-4469-bc71-a63b06b020c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51dbf47189fc4f3c90eadc4887a831b8', 'request_id': 'req-cc84ef6b-037a-4469-bc71-a63b06b020c3'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:50.795809 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-cc84ef6b-037a-4469-bc71-a63b06b020c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51dbf47189fc4f3c90eadc4887a831b8', 'request_id': 'req-cc84ef6b-037a-4469-bc71-a63b06b020c3'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:50.806976 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-cc84ef6b-037a-4469-bc71-a63b06b020c3 admin admin] The token cache is being invalidate because project 51dbf47189fc4f3c90eadc4887a831b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:12:50.807269 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-cc84ef6b-037a-4469-bc71-a63b06b020c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:50.807902 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-cc84ef6b-037a-4469-bc71-a63b06b020c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:50.809706 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1266/2527] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:12:50 2022] DELETE /identity/v3/projects/51dbf47189fc4f3c90eadc4887a831b8 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:12:50.814725 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275999b-702c-48d3-b650-3bc8c211452b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:50.815017 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275999b-702c-48d3-b650-3bc8c211452b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:50.815303 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275999b-702c-48d3-b650-3bc8c211452b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:50.835141 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1275999b-702c-48d3-b650-3bc8c211452b None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:12:50.836379 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1275999b-702c-48d3-b650-3bc8c211452b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:12:50.854233 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:12:50.897384 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1262/2528] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:12:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:12:50.905940 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419dbb5-e444-481a-adc7-86a25667ad07 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:50.906436 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419dbb5-e444-481a-adc7-86a25667ad07 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:50.906542 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419dbb5-e444-481a-adc7-86a25667ad07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:50.927812 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-4419dbb5-e444-481a-adc7-86a25667ad07 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:12:50.933272 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-4419dbb5-e444-481a-adc7-86a25667ad07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:12:50.936882 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1267/2529] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:12:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:12:50.941104 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275999b-702c-48d3-b650-3bc8c211452b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:50.951481 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275999b-702c-48d3-b650-3bc8c211452b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:50.956093 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1d8fb-dafc-44ee-94a5-a32ea9e1f8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:50.956730 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1d8fb-dafc-44ee-94a5-a32ea9e1f8d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:50.957013 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1d8fb-dafc-44ee-94a5-a32ea9e1f8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:50.957292 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1d8fb-dafc-44ee-94a5-a32ea9e1f8d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:50.957986 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b1d8fb-dafc-44ee-94a5-a32ea9e1f8d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:12:50.958554 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b1d8fb-dafc-44ee-94a5-a32ea9e1f8d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:50.959885 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b1d8fb-dafc-44ee-94a5-a32ea9e1f8d0 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:50.971977 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1263/2530] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:12:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:12:50.976214 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4419dbb5-e444-481a-adc7-86a25667ad07 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:50.986208 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4419dbb5-e444-481a-adc7-86a25667ad07 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:50.990877 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc92d3a-759a-408c-9261-97693eec0f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:50.991521 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc92d3a-759a-408c-9261-97693eec0f8e admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:50.991738 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc92d3a-759a-408c-9261-97693eec0f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:50.992019 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc92d3a-759a-408c-9261-97693eec0f8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:50.992689 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc92d3a-759a-408c-9261-97693eec0f8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:50.994584 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc92d3a-759a-408c-9261-97693eec0f8e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:51.006529 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:12:51.085745 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1268/2531] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:12:50 2022] POST /identity/v3/projects => generated 405 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:12:51.087301 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1d8fb-dafc-44ee-94a5-a32ea9e1f8d0 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:51.099093 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1d8fb-dafc-44ee-94a5-a32ea9e1f8d0 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:51.160097 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1f4b37-cb61-4597-b237-9ee2ae9a6dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:51.160722 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f4b37-cb61-4597-b237-9ee2ae9a6dea admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:51.161006 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f4b37-cb61-4597-b237-9ee2ae9a6dea admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:51.161280 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f4b37-cb61-4597-b237-9ee2ae9a6dea admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:51.161908 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1f4b37-cb61-4597-b237-9ee2ae9a6dea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:51.163843 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1f4b37-cb61-4597-b237-9ee2ae9a6dea admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:51.247558 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1264/2532] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:12:51 2022] POST /identity/v3/users => generated 303 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:12:51.251612 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc92d3a-759a-408c-9261-97693eec0f8e admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:51.263274 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc92d3a-759a-408c-9261-97693eec0f8e admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:51.267942 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae827d9-815e-4fdd-925b-1b6b2837a274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:51.268560 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae827d9-815e-4fdd-925b-1b6b2837a274 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:51.268845 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae827d9-815e-4fdd-925b-1b6b2837a274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:51.269119 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae827d9-815e-4fdd-925b-1b6b2837a274 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:51.269754 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae827d9-815e-4fdd-925b-1b6b2837a274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:51.271350 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae827d9-815e-4fdd-925b-1b6b2837a274 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:51.282399 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1269/2533] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:12:51.286350 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1f4b37-cb61-4597-b237-9ee2ae9a6dea admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:51.296956 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1f4b37-cb61-4597-b237-9ee2ae9a6dea admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:51.301773 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e093f7ba-ce98-446d-9162-64b3747762cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:51.302418 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093f7ba-ce98-446d-9162-64b3747762cc admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:51.302725 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093f7ba-ce98-446d-9162-64b3747762cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:51.303098 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093f7ba-ce98-446d-9162-64b3747762cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:51.303616 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e093f7ba-ce98-446d-9162-64b3747762cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:51.305427 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e093f7ba-ce98-446d-9162-64b3747762cc admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:51.315716 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1265/2534] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:12:51.320019 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae827d9-815e-4fdd-925b-1b6b2837a274 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:51.330608 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae827d9-815e-4fdd-925b-1b6b2837a274 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:51.335190 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71922ec9-09cc-4478-b30c-95f0d9a752a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:51.335824 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71922ec9-09cc-4478-b30c-95f0d9a752a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:51.336157 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71922ec9-09cc-4478-b30c-95f0d9a752a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:51.336435 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71922ec9-09cc-4478-b30c-95f0d9a752a5 admin admin] PATH_INFO: `/v3/projects/1e64f289235f41c2adb096727f41813b/users/8beecd5020ac4f03b7afc5bc94865316/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:51.354309 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71922ec9-09cc-4478-b30c-95f0d9a752a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e64f289235f41c2adb096727f41813b, user_id=8beecd5020ac4f03b7afc5bc94865316, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:51.357267 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71922ec9-09cc-4478-b30c-95f0d9a752a5 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:51.435635 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1270/2535] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:12:51 2022] PUT /identity/v3/projects/1e64f289235f41c2adb096727f41813b/users/8beecd5020ac4f03b7afc5bc94865316/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:12:51.445890 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ec316-bb17-44ed-8014-a14743739e68 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:51.447751 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ec316-bb17-44ed-8014-a14743739e68 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:51.447751 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ec316-bb17-44ed-8014-a14743739e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:51.476200 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-4a2ec316-bb17-44ed-8014-a14743739e68 None None] MFA Rules not processed for user `8beecd5020ac4f03b7afc5bc94865316`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:12:51.477457 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4a2ec316-bb17-44ed-8014-a14743739e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:12:51.541060 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1266/2536] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:12:51 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:12:51.545833 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71922ec9-09cc-4478-b30c-95f0d9a752a5 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:51.607491 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71922ec9-09cc-4478-b30c-95f0d9a752a5 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:51.612414 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd8091e-4fb3-4e9d-81d7-4b98f6583ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:51.613047 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8091e-4fb3-4e9d-81d7-4b98f6583ed4 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:51.613330 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8091e-4fb3-4e9d-81d7-4b98f6583ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:51.613607 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8091e-4fb3-4e9d-81d7-4b98f6583ed4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:51.614245 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd8091e-4fb3-4e9d-81d7-4b98f6583ed4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:51.616157 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd8091e-4fb3-4e9d-81d7-4b98f6583ed4 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:51.751067 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1271/2537] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:12:51 2022] POST /identity/v3/projects => generated 405 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:12:51.755260 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ec316-bb17-44ed-8014-a14743739e68 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:51.767229 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ec316-bb17-44ed-8014-a14743739e68 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:51.779477 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:12:51.822877 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a75a4b-fde4-49de-ad80-a13eab9afe93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:51.823479 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a75a4b-fde4-49de-ad80-a13eab9afe93 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:51.823760 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a75a4b-fde4-49de-ad80-a13eab9afe93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:51.824035 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a75a4b-fde4-49de-ad80-a13eab9afe93 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:51.825304 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a75a4b-fde4-49de-ad80-a13eab9afe93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:51.827214 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a75a4b-fde4-49de-ad80-a13eab9afe93 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:51.898361 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1267/2538] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:12:51 2022] POST /identity/v3/users => generated 309 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:12:51.902311 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd8091e-4fb3-4e9d-81d7-4b98f6583ed4 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:51.913522 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd8091e-4fb3-4e9d-81d7-4b98f6583ed4 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:51.919395 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0098f64-6279-4eb3-a8c4-3616d3bbc082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:51.920026 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0098f64-6279-4eb3-a8c4-3616d3bbc082 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:51.920306 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0098f64-6279-4eb3-a8c4-3616d3bbc082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:51.920614 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0098f64-6279-4eb3-a8c4-3616d3bbc082 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:51.921250 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0098f64-6279-4eb3-a8c4-3616d3bbc082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:51.922844 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0098f64-6279-4eb3-a8c4-3616d3bbc082 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:51.933270 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1272/2539] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:12:51.938061 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a75a4b-fde4-49de-ad80-a13eab9afe93 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:51.948096 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a75a4b-fde4-49de-ad80-a13eab9afe93 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:51.952770 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c57c0d-f8d8-4814-8ccc-425f3e0308dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:51.953415 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c57c0d-f8d8-4814-8ccc-425f3e0308dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:51.953698 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c57c0d-f8d8-4814-8ccc-425f3e0308dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:51.953977 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c57c0d-f8d8-4814-8ccc-425f3e0308dd admin admin] PATH_INFO: `/v3/projects/b6089cc199f94de1b5a0dc3e9755c8a9/users/9f1c8614cb52476fb1547820b0451b3f/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:51.968836 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c57c0d-f8d8-4814-8ccc-425f3e0308dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6089cc199f94de1b5a0dc3e9755c8a9, user_id=9f1c8614cb52476fb1547820b0451b3f, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:51.971543 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c57c0d-f8d8-4814-8ccc-425f3e0308dd admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:52.029172 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1268/2540] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:12:51 2022] PUT /identity/v3/projects/b6089cc199f94de1b5a0dc3e9755c8a9/users/9f1c8614cb52476fb1547820b0451b3f/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:12:52.033120 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0098f64-6279-4eb3-a8c4-3616d3bbc082 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:52.043550 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0098f64-6279-4eb3-a8c4-3616d3bbc082 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:52.100103 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc05a8-2529-49f8-81e7-5019ba50edcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:52.100748 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc05a8-2529-49f8-81e7-5019ba50edcd admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:52.101036 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc05a8-2529-49f8-81e7-5019ba50edcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:52.101314 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc05a8-2529-49f8-81e7-5019ba50edcd admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:52.101955 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbc05a8-2529-49f8-81e7-5019ba50edcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:52.103586 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbc05a8-2529-49f8-81e7-5019ba50edcd admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:52.113738 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1273/2541] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:12:52 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:12:52.118073 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c57c0d-f8d8-4814-8ccc-425f3e0308dd admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:52.129072 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c57c0d-f8d8-4814-8ccc-425f3e0308dd admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:52.133640 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdadc3f-e65d-4e06-ab43-e45504552ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:52.134281 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdadc3f-e65d-4e06-ab43-e45504552ed7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:52.134560 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdadc3f-e65d-4e06-ab43-e45504552ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:52.134836 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdadc3f-e65d-4e06-ab43-e45504552ed7 admin admin] PATH_INFO: `/v3/projects/b6089cc199f94de1b5a0dc3e9755c8a9/users/9f1c8614cb52476fb1547820b0451b3f/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:52.137258 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdadc3f-e65d-4e06-ab43-e45504552ed7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6089cc199f94de1b5a0dc3e9755c8a9, user_id=9f1c8614cb52476fb1547820b0451b3f, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:52.139988 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdadc3f-e65d-4e06-ab43-e45504552ed7 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:52.150004 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1269/2542] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:12:52 2022] PUT /identity/v3/projects/b6089cc199f94de1b5a0dc3e9755c8a9/users/9f1c8614cb52476fb1547820b0451b3f/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:12:52.161655 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d2229-888d-447c-b1f8-782bb5691a9e None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:52.162055 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d2229-888d-447c-b1f8-782bb5691a9e None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:52.162401 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d2229-888d-447c-b1f8-782bb5691a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:52.192292 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-418d2229-888d-447c-b1f8-782bb5691a9e None None] MFA Rules not processed for user `9f1c8614cb52476fb1547820b0451b3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:12:52.193543 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-418d2229-888d-447c-b1f8-782bb5691a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:12:52.206594 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:12:52.272282 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1274/2543] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:12:52 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:12:52.283570 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdadc3f-e65d-4e06-ab43-e45504552ed7 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:52.290952 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ffdadc3f-e65d-4e06-ab43-e45504552ed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:12:52.347058 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdadc3f-e65d-4e06-ab43-e45504552ed7 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:52.350806 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0faa18c-a442-4420-8ca8-0148609d11eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:52.351416 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0faa18c-a442-4420-8ca8-0148609d11eb service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:52.354019 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0faa18c-a442-4420-8ca8-0148609d11eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:52.354019 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0faa18c-a442-4420-8ca8-0148609d11eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:52.362695 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0faa18c-a442-4420-8ca8-0148609d11eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:52.364127 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0faa18c-a442-4420-8ca8-0148609d11eb service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:52.374170 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1270/2544] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:12:52 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:12:52.545646 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d2229-888d-447c-b1f8-782bb5691a9e None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:52.615005 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d2229-888d-447c-b1f8-782bb5691a9e None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:52.620051 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82774a2a-a18e-41ce-9bb6-032e22022609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:52.620845 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82774a2a-a18e-41ce-9bb6-032e22022609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:52.621142 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82774a2a-a18e-41ce-9bb6-032e22022609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:52.622021 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82774a2a-a18e-41ce-9bb6-032e22022609 admin admin] PATH_INFO: `/v3/users/8beecd5020ac4f03b7afc5bc94865316` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:52.624257 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82774a2a-a18e-41ce-9bb6-032e22022609 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8beecd5020ac4f03b7afc5bc94865316)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:52.626234 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82774a2a-a18e-41ce-9bb6-032e22022609 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:52.815248 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-82774a2a-a18e-41ce-9bb6-032e22022609 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8beecd5020ac4f03b7afc5bc94865316'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:52.815904 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-82774a2a-a18e-41ce-9bb6-032e22022609 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8beecd5020ac4f03b7afc5bc94865316'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:52.892471 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-82774a2a-a18e-41ce-9bb6-032e22022609 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8beecd5020ac4f03b7afc5bc94865316'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:52.893116 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-82774a2a-a18e-41ce-9bb6-032e22022609 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8beecd5020ac4f03b7afc5bc94865316'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:52.906812 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-82774a2a-a18e-41ce-9bb6-032e22022609 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8beecd5020ac4f03b7afc5bc94865316'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:52.919952 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1275/2545] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:12:52 2022] DELETE /identity/v3/users/8beecd5020ac4f03b7afc5bc94865316 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:12:52.924892 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52340183-b37a-4f6f-ac66-af3d2c685fbe None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:52.926380 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52340183-b37a-4f6f-ac66-af3d2c685fbe None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:52.926380 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52340183-b37a-4f6f-ac66-af3d2c685fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:52.945947 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-52340183-b37a-4f6f-ac66-af3d2c685fbe None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:12:52.947149 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-52340183-b37a-4f6f-ac66-af3d2c685fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:12:53.015776 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1271/2546] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:12:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:12:53.025729 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82774a2a-a18e-41ce-9bb6-032e22022609 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:53.027188 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-82774a2a-a18e-41ce-9bb6-032e22022609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:12:53.085904 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:12:53.108877 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82774a2a-a18e-41ce-9bb6-032e22022609 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:53.112417 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d31e7-3704-46ec-82de-e405f7c48168 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:53.113040 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d31e7-3704-46ec-82de-e405f7c48168 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:53.113317 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d31e7-3704-46ec-82de-e405f7c48168 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:53.113637 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d31e7-3704-46ec-82de-e405f7c48168 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:53.123817 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251d31e7-3704-46ec-82de-e405f7c48168 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:53.125347 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251d31e7-3704-46ec-82de-e405f7c48168 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:53.134775 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1276/2547] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:12:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:12:53.540040 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52340183-b37a-4f6f-ac66-af3d2c685fbe None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:53.541487 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-52340183-b37a-4f6f-ac66-af3d2c685fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:12:53.554380 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52340183-b37a-4f6f-ac66-af3d2c685fbe None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:53.560122 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d771bb45-f093-4791-a030-43f22e515858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:53.560819 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771bb45-f093-4791-a030-43f22e515858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:53.561102 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771bb45-f093-4791-a030-43f22e515858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:53.561385 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771bb45-f093-4791-a030-43f22e515858 admin admin] PATH_INFO: `/v3/projects/1e64f289235f41c2adb096727f41813b` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:53.562559 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d771bb45-f093-4791-a030-43f22e515858 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e64f289235f41c2adb096727f41813b)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:53.564536 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d771bb45-f093-4791-a030-43f22e515858 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:53.573968 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:12:53.658525 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d771bb45-f093-4791-a030-43f22e515858 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e64f289235f41c2adb096727f41813b', 'request_id': 'req-d771bb45-f093-4791-a030-43f22e515858'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:53.703890 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d771bb45-f093-4791-a030-43f22e515858 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e64f289235f41c2adb096727f41813b', 'request_id': 'req-d771bb45-f093-4791-a030-43f22e515858'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:53.709540 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d771bb45-f093-4791-a030-43f22e515858 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e64f289235f41c2adb096727f41813b', 'request_id': 'req-d771bb45-f093-4791-a030-43f22e515858'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:53.721091 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d771bb45-f093-4791-a030-43f22e515858 admin admin] The token cache is being invalidate because project 1e64f289235f41c2adb096727f41813b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:12:53.721376 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d771bb45-f093-4791-a030-43f22e515858 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:53.721992 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d771bb45-f093-4791-a030-43f22e515858 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:53.723604 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1272/2548] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:12:53 2022] DELETE /identity/v3/projects/1e64f289235f41c2adb096727f41813b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:12:53.727761 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d31e7-3704-46ec-82de-e405f7c48168 service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:53.738206 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d31e7-3704-46ec-82de-e405f7c48168 service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:53.796353 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b212d0-f99e-4653-845f-2bc80b957fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:53.797004 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b212d0-f99e-4653-845f-2bc80b957fee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:53.797285 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b212d0-f99e-4653-845f-2bc80b957fee admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:53.797558 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b212d0-f99e-4653-845f-2bc80b957fee admin admin] PATH_INFO: `/v3/users/9f1c8614cb52476fb1547820b0451b3f` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:53.798660 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b212d0-f99e-4653-845f-2bc80b957fee admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f1c8614cb52476fb1547820b0451b3f)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:53.800626 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b212d0-f99e-4653-845f-2bc80b957fee admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:53.951625 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-75b212d0-f99e-4653-845f-2bc80b957fee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f1c8614cb52476fb1547820b0451b3f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:53.952296 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-75b212d0-f99e-4653-845f-2bc80b957fee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f1c8614cb52476fb1547820b0451b3f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:54.023322 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-75b212d0-f99e-4653-845f-2bc80b957fee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f1c8614cb52476fb1547820b0451b3f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:54.023927 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-75b212d0-f99e-4653-845f-2bc80b957fee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f1c8614cb52476fb1547820b0451b3f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:54.036534 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-75b212d0-f99e-4653-845f-2bc80b957fee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f1c8614cb52476fb1547820b0451b3f'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:54.049020 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1277/2549] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:12:53 2022] DELETE /identity/v3/users/9f1c8614cb52476fb1547820b0451b3f => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:12:54.477009 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d771bb45-f093-4791-a030-43f22e515858 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:54.478435 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d771bb45-f093-4791-a030-43f22e515858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:12:54.492094 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d771bb45-f093-4791-a030-43f22e515858 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:54.558420 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ddedf-4a71-437c-877a-ddc7a523a714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:54.559905 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ddedf-4a71-437c-877a-ddc7a523a714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:54.560190 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ddedf-4a71-437c-877a-ddc7a523a714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:54.560479 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ddedf-4a71-437c-877a-ddc7a523a714 admin admin] PATH_INFO: `/v3/projects/b6089cc199f94de1b5a0dc3e9755c8a9` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:54.561615 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486ddedf-4a71-437c-877a-ddc7a523a714 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6089cc199f94de1b5a0dc3e9755c8a9)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:54.563503 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486ddedf-4a71-437c-877a-ddc7a523a714 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:54.677821 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-486ddedf-4a71-437c-877a-ddc7a523a714 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6089cc199f94de1b5a0dc3e9755c8a9', 'request_id': 'req-486ddedf-4a71-437c-877a-ddc7a523a714'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:54.743922 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-486ddedf-4a71-437c-877a-ddc7a523a714 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6089cc199f94de1b5a0dc3e9755c8a9', 'request_id': 'req-486ddedf-4a71-437c-877a-ddc7a523a714'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:54.751035 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-486ddedf-4a71-437c-877a-ddc7a523a714 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6089cc199f94de1b5a0dc3e9755c8a9', 'request_id': 'req-486ddedf-4a71-437c-877a-ddc7a523a714'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:54.763517 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-486ddedf-4a71-437c-877a-ddc7a523a714 admin admin] The token cache is being invalidate because project b6089cc199f94de1b5a0dc3e9755c8a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:12:54.763810 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-486ddedf-4a71-437c-877a-ddc7a523a714 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:54.764442 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-486ddedf-4a71-437c-877a-ddc7a523a714 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:12:54.765943 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1273/2550] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:12:54 2022] DELETE /identity/v3/projects/b6089cc199f94de1b5a0dc3e9755c8a9 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:12:54.772366 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b0bfe-f72b-4dce-8c67-2dcd14df9ab6 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:54.773398 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b0bfe-f72b-4dce-8c67-2dcd14df9ab6 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:54.773398 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b0bfe-f72b-4dce-8c67-2dcd14df9ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:54.793791 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-231b0bfe-f72b-4dce-8c67-2dcd14df9ab6 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:12:54.795006 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-231b0bfe-f72b-4dce-8c67-2dcd14df9ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:12:54.842541 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:12:54.854349 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1278/2551] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:12:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:12:54.863212 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b764fa-b61a-4779-95d2-08d79ead2e87 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:54.863498 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b764fa-b61a-4779-95d2-08d79ead2e87 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:54.863774 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b764fa-b61a-4779-95d2-08d79ead2e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:54.885738 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-63b764fa-b61a-4779-95d2-08d79ead2e87 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:12:54.887423 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-63b764fa-b61a-4779-95d2-08d79ead2e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:12:54.895329 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1274/2552] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:12:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:12:54.899547 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b0bfe-f72b-4dce-8c67-2dcd14df9ab6 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:54.909721 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b0bfe-f72b-4dce-8c67-2dcd14df9ab6 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:54.914685 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8714ed-7baa-4e1c-88d5-b068fc21b468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:54.915285 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8714ed-7baa-4e1c-88d5-b068fc21b468 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:54.915566 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8714ed-7baa-4e1c-88d5-b068fc21b468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:54.915840 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8714ed-7baa-4e1c-88d5-b068fc21b468 admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:54.916525 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8714ed-7baa-4e1c-88d5-b068fc21b468 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:12:54.916835 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8714ed-7baa-4e1c-88d5-b068fc21b468 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:54.919102 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8714ed-7baa-4e1c-88d5-b068fc21b468 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:54.930357 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1279/2553] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:12:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:12:54.935355 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b764fa-b61a-4779-95d2-08d79ead2e87 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:54.944574 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b764fa-b61a-4779-95d2-08d79ead2e87 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:54.949185 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b583d47-f624-409d-af94-8c5b67047532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:54.949969 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b583d47-f624-409d-af94-8c5b67047532 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:54.950253 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b583d47-f624-409d-af94-8c5b67047532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:54.950528 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b583d47-f624-409d-af94-8c5b67047532 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:54.951155 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b583d47-f624-409d-af94-8c5b67047532 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:54.953079 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b583d47-f624-409d-af94-8c5b67047532 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:55.023703 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:12:55.030885 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1275/2554] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:12:54 2022] POST /identity/v3/projects => generated 401 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:12:55.037021 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8714ed-7baa-4e1c-88d5-b068fc21b468 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:55.045978 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8714ed-7baa-4e1c-88d5-b068fc21b468 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:55.101608 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3297e87f-81ef-45f0-94c7-ad58847e2dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:55.102249 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297e87f-81ef-45f0-94c7-ad58847e2dcc admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:55.102532 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297e87f-81ef-45f0-94c7-ad58847e2dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:55.102806 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297e87f-81ef-45f0-94c7-ad58847e2dcc admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:55.103426 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3297e87f-81ef-45f0-94c7-ad58847e2dcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:55.105377 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3297e87f-81ef-45f0-94c7-ad58847e2dcc admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:55.195263 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1280/2555] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:12:55 2022] POST /identity/v3/users => generated 301 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:12:55.199542 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b583d47-f624-409d-af94-8c5b67047532 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:55.210190 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b583d47-f624-409d-af94-8c5b67047532 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:55.214903 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3a9d5-1c72-41f1-9b37-5c6fa71debb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:55.215709 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3a9d5-1c72-41f1-9b37-5c6fa71debb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:55.215991 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3a9d5-1c72-41f1-9b37-5c6fa71debb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:55.216265 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3a9d5-1c72-41f1-9b37-5c6fa71debb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:55.216943 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e3a9d5-1c72-41f1-9b37-5c6fa71debb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:55.218551 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e3a9d5-1c72-41f1-9b37-5c6fa71debb6 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:55.230908 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1276/2556] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:12:55.234988 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3297e87f-81ef-45f0-94c7-ad58847e2dcc admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:55.245157 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3297e87f-81ef-45f0-94c7-ad58847e2dcc admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:55.250397 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60805f1-e6de-4746-a60c-291a70770b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:55.250991 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60805f1-e6de-4746-a60c-291a70770b4f admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:55.251271 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60805f1-e6de-4746-a60c-291a70770b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:55.251545 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60805f1-e6de-4746-a60c-291a70770b4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:55.252213 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60805f1-e6de-4746-a60c-291a70770b4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:55.253838 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60805f1-e6de-4746-a60c-291a70770b4f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:55.264399 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1281/2557] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:12:55.268438 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3a9d5-1c72-41f1-9b37-5c6fa71debb6 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:55.278787 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3a9d5-1c72-41f1-9b37-5c6fa71debb6 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:55.283230 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973e5c9-79ff-4c5a-9f25-704a295763ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:55.283867 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973e5c9-79ff-4c5a-9f25-704a295763ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:55.284145 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973e5c9-79ff-4c5a-9f25-704a295763ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:55.284414 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973e5c9-79ff-4c5a-9f25-704a295763ed admin admin] PATH_INFO: `/v3/projects/d90b995956e847b79b590539114da32e/users/21b6c091420549669b2d49808fbe13c5/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:55.299431 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973e5c9-79ff-4c5a-9f25-704a295763ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=d90b995956e847b79b590539114da32e, user_id=21b6c091420549669b2d49808fbe13c5, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:55.302099 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973e5c9-79ff-4c5a-9f25-704a295763ed admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:55.361364 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1277/2558] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:12:55 2022] PUT /identity/v3/projects/d90b995956e847b79b590539114da32e/users/21b6c091420549669b2d49808fbe13c5/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:12:55.372961 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7dcd3-fb1b-499c-baff-f35c4de0a7cd None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:55.373247 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7dcd3-fb1b-499c-baff-f35c4de0a7cd None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:55.373524 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7dcd3-fb1b-499c-baff-f35c4de0a7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:55.401392 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-b8a7dcd3-fb1b-499c-baff-f35c4de0a7cd None None] MFA Rules not processed for user `21b6c091420549669b2d49808fbe13c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:12:55.403173 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b8a7dcd3-fb1b-499c-baff-f35c4de0a7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:12:55.465694 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1282/2559] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:12:55 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:12:55.470946 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973e5c9-79ff-4c5a-9f25-704a295763ed admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:55.532805 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973e5c9-79ff-4c5a-9f25-704a295763ed admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:55.537325 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8686be-d89c-44b8-8b7b-0e5e0cc7c0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:55.537931 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8686be-d89c-44b8-8b7b-0e5e0cc7c0e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:55.538209 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8686be-d89c-44b8-8b7b-0e5e0cc7c0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:55.538483 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8686be-d89c-44b8-8b7b-0e5e0cc7c0e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:55.539114 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8686be-d89c-44b8-8b7b-0e5e0cc7c0e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:55.541059 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8686be-d89c-44b8-8b7b-0e5e0cc7c0e4 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:55.637146 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1278/2560] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:12:55 2022] POST /identity/v3/projects => generated 401 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:12:55.642299 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a7dcd3-fb1b-499c-baff-f35c4de0a7cd None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:55.652965 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a7dcd3-fb1b-499c-baff-f35c4de0a7cd None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:55.699474 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:12:55.714561 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d39868-6ede-4ae5-b158-f63d56bb3b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:55.715295 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d39868-6ede-4ae5-b158-f63d56bb3b79 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:55.717789 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d39868-6ede-4ae5-b158-f63d56bb3b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:55.718272 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d39868-6ede-4ae5-b158-f63d56bb3b79 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:55.718993 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d39868-6ede-4ae5-b158-f63d56bb3b79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:55.721171 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d39868-6ede-4ae5-b158-f63d56bb3b79 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:55.790889 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1283/2561] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:12:55 2022] POST /identity/v3/users => generated 307 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:12:55.794028 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8686be-d89c-44b8-8b7b-0e5e0cc7c0e4 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:55.805998 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8686be-d89c-44b8-8b7b-0e5e0cc7c0e4 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:55.811929 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2107a-73ab-4944-8a10-a6a3a5b6ce75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:55.812715 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2107a-73ab-4944-8a10-a6a3a5b6ce75 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:55.813071 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2107a-73ab-4944-8a10-a6a3a5b6ce75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:55.813411 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2107a-73ab-4944-8a10-a6a3a5b6ce75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:55.814823 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff2107a-73ab-4944-8a10-a6a3a5b6ce75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:55.815996 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff2107a-73ab-4944-8a10-a6a3a5b6ce75 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:55.828050 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1279/2562] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:12:55.832294 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d39868-6ede-4ae5-b158-f63d56bb3b79 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:55.842613 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d39868-6ede-4ae5-b158-f63d56bb3b79 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:55.847642 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532daa5-2944-4883-aec9-a79539f2a87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:55.848301 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532daa5-2944-4883-aec9-a79539f2a87b admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:55.848614 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532daa5-2944-4883-aec9-a79539f2a87b admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:55.848888 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532daa5-2944-4883-aec9-a79539f2a87b admin admin] PATH_INFO: `/v3/projects/fe50c2023f4b4bb9908b7c38465fc3d2/users/eb20d1b5256545d5a6897acb7461cbda/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:55.865236 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1532daa5-2944-4883-aec9-a79539f2a87b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe50c2023f4b4bb9908b7c38465fc3d2, user_id=eb20d1b5256545d5a6897acb7461cbda, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:55.868157 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1532daa5-2944-4883-aec9-a79539f2a87b admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:55.939606 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1284/2563] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:12:55 2022] PUT /identity/v3/projects/fe50c2023f4b4bb9908b7c38465fc3d2/users/eb20d1b5256545d5a6897acb7461cbda/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:12:55.943618 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2107a-73ab-4944-8a10-a6a3a5b6ce75 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:55.953756 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2107a-73ab-4944-8a10-a6a3a5b6ce75 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:56.011419 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a4ac50-0edd-4f13-bd7f-beb1585ebfa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:56.012019 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4ac50-0edd-4f13-bd7f-beb1585ebfa9 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:56.012308 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4ac50-0edd-4f13-bd7f-beb1585ebfa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:56.012627 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4ac50-0edd-4f13-bd7f-beb1585ebfa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:56.014030 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a4ac50-0edd-4f13-bd7f-beb1585ebfa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:56.015470 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a4ac50-0edd-4f13-bd7f-beb1585ebfa9 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:56.025135 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1280/2564] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:12:56.029677 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532daa5-2944-4883-aec9-a79539f2a87b admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:56.039748 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532daa5-2944-4883-aec9-a79539f2a87b admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:56.044660 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7610587c-1f1d-42b4-b133-6ab9d40cbd96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:56.045247 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7610587c-1f1d-42b4-b133-6ab9d40cbd96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:56.045522 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7610587c-1f1d-42b4-b133-6ab9d40cbd96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:56.045795 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7610587c-1f1d-42b4-b133-6ab9d40cbd96 admin admin] PATH_INFO: `/v3/projects/fe50c2023f4b4bb9908b7c38465fc3d2/users/eb20d1b5256545d5a6897acb7461cbda/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:56.047695 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7610587c-1f1d-42b4-b133-6ab9d40cbd96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe50c2023f4b4bb9908b7c38465fc3d2, user_id=eb20d1b5256545d5a6897acb7461cbda, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:56.050343 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7610587c-1f1d-42b4-b133-6ab9d40cbd96 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:56.060767 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1285/2565] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:12:56 2022] PUT /identity/v3/projects/fe50c2023f4b4bb9908b7c38465fc3d2/users/eb20d1b5256545d5a6897acb7461cbda/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:12:56.070648 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896d81f-7f58-4b89-8de9-56eb5a24566a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:56.070934 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896d81f-7f58-4b89-8de9-56eb5a24566a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:56.071209 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896d81f-7f58-4b89-8de9-56eb5a24566a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:56.099792 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-d896d81f-7f58-4b89-8de9-56eb5a24566a None None] MFA Rules not processed for user `eb20d1b5256545d5a6897acb7461cbda`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:12:56.101504 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d896d81f-7f58-4b89-8de9-56eb5a24566a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:12:56.121013 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:12:56.171162 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1281/2566] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:12:56 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:12:56.182303 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7610587c-1f1d-42b4-b133-6ab9d40cbd96 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:56.184277 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-7610587c-1f1d-42b4-b133-6ab9d40cbd96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:12:56.243466 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7610587c-1f1d-42b4-b133-6ab9d40cbd96 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:56.247073 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd4e284-b177-45a0-b800-6b6afc4bf6b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:56.248173 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4e284-b177-45a0-b800-6b6afc4bf6b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:56.248488 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4e284-b177-45a0-b800-6b6afc4bf6b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:56.248822 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4e284-b177-45a0-b800-6b6afc4bf6b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:56.257651 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd4e284-b177-45a0-b800-6b6afc4bf6b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:56.259079 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd4e284-b177-45a0-b800-6b6afc4bf6b2 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:56.269386 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1286/2567] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:12:56 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:12:56.473034 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896d81f-7f58-4b89-8de9-56eb5a24566a None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:12:56.483362 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896d81f-7f58-4b89-8de9-56eb5a24566a None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:12:56.486948 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1824f519-e7f7-4419-9b2e-10aeec67c8c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:12:56.487541 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824f519-e7f7-4419-9b2e-10aeec67c8c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:12:56.488008 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824f519-e7f7-4419-9b2e-10aeec67c8c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:12:56.488301 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824f519-e7f7-4419-9b2e-10aeec67c8c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:12:56.546285 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1824f519-e7f7-4419-9b2e-10aeec67c8c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:12:56.547716 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1824f519-e7f7-4419-9b2e-10aeec67c8c6 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:12:56.556525 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1282/2568] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:12:56 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:13:01.598954 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd4e284-b177-45a0-b800-6b6afc4bf6b2 service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:13:01.662382 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd4e284-b177-45a0-b800-6b6afc4bf6b2 service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:13:01.666918 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603a34c-9c07-4de7-909d-4868e95fc1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:13:01.667545 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603a34c-9c07-4de7-909d-4868e95fc1a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:01.667827 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603a34c-9c07-4de7-909d-4868e95fc1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:01.668099 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603a34c-9c07-4de7-909d-4868e95fc1a2 admin admin] PATH_INFO: `/v3/users/21b6c091420549669b2d49808fbe13c5` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:01.669327 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8603a34c-9c07-4de7-909d-4868e95fc1a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21b6c091420549669b2d49808fbe13c5)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:13:01.671317 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8603a34c-9c07-4de7-909d-4868e95fc1a2 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:13:01.853183 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:13:01.854720 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-8603a34c-9c07-4de7-909d-4868e95fc1a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21b6c091420549669b2d49808fbe13c5'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:13:01.855393 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-8603a34c-9c07-4de7-909d-4868e95fc1a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21b6c091420549669b2d49808fbe13c5'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:13:01.914882 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-8603a34c-9c07-4de7-909d-4868e95fc1a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21b6c091420549669b2d49808fbe13c5'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:13:01.915511 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-8603a34c-9c07-4de7-909d-4868e95fc1a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21b6c091420549669b2d49808fbe13c5'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:13:01.928022 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-8603a34c-9c07-4de7-909d-4868e95fc1a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21b6c091420549669b2d49808fbe13c5'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:13:01.940575 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1287/2569] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:13:01 2022] DELETE /identity/v3/users/21b6c091420549669b2d49808fbe13c5 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:13:01.945088 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa0754e-182e-4581-ba9d-8ded646c08ad None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:01.945967 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa0754e-182e-4581-ba9d-8ded646c08ad None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:01.946249 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa0754e-182e-4581-ba9d-8ded646c08ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:01.965669 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-9aa0754e-182e-4581-ba9d-8ded646c08ad None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:13:01.966894 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9aa0754e-182e-4581-ba9d-8ded646c08ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:13:02.024187 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1283/2570] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:13:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:13:02.034158 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603a34c-9c07-4de7-909d-4868e95fc1a2 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:13:02.035568 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8603a34c-9c07-4de7-909d-4868e95fc1a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:13:02.092406 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603a34c-9c07-4de7-909d-4868e95fc1a2 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:13:02.096079 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a0dd5-7cff-47fc-84f1-47c950499bc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:13:02.096717 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a0dd5-7cff-47fc-84f1-47c950499bc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:02.097000 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a0dd5-7cff-47fc-84f1-47c950499bc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:02.097278 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a0dd5-7cff-47fc-84f1-47c950499bc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:02.106268 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120a0dd5-7cff-47fc-84f1-47c950499bc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:13:02.107991 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120a0dd5-7cff-47fc-84f1-47c950499bc4 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:13:02.118448 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1288/2571] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:13:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:13:02.564791 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa0754e-182e-4581-ba9d-8ded646c08ad None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:13:02.565267 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9aa0754e-182e-4581-ba9d-8ded646c08ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:13:02.579485 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa0754e-182e-4581-ba9d-8ded646c08ad None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:13:02.585253 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c7f5a-a13d-4793-9816-3572b51de295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:13:02.586424 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c7f5a-a13d-4793-9816-3572b51de295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:02.586424 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c7f5a-a13d-4793-9816-3572b51de295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:02.586737 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c7f5a-a13d-4793-9816-3572b51de295 admin admin] PATH_INFO: `/v3/projects/d90b995956e847b79b590539114da32e` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:02.588046 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c7f5a-a13d-4793-9816-3572b51de295 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d90b995956e847b79b590539114da32e)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:13:02.591424 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c7f5a-a13d-4793-9816-3572b51de295 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:13:02.734514 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1b9c7f5a-a13d-4793-9816-3572b51de295 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd90b995956e847b79b590539114da32e', 'request_id': 'req-1b9c7f5a-a13d-4793-9816-3572b51de295'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:13:02.737017 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:13:02.812942 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1b9c7f5a-a13d-4793-9816-3572b51de295 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd90b995956e847b79b590539114da32e', 'request_id': 'req-1b9c7f5a-a13d-4793-9816-3572b51de295'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:13:02.820909 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1b9c7f5a-a13d-4793-9816-3572b51de295 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd90b995956e847b79b590539114da32e', 'request_id': 'req-1b9c7f5a-a13d-4793-9816-3572b51de295'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:13:02.832375 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1b9c7f5a-a13d-4793-9816-3572b51de295 admin admin] The token cache is being invalidate because project d90b995956e847b79b590539114da32e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:13:02.834257 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1b9c7f5a-a13d-4793-9816-3572b51de295 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:13:02.834257 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1b9c7f5a-a13d-4793-9816-3572b51de295 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:13:02.835099 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1284/2572] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:13:02 2022] DELETE /identity/v3/projects/d90b995956e847b79b590539114da32e => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:13:02.839324 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a0dd5-7cff-47fc-84f1-47c950499bc4 service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:13:02.851192 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a0dd5-7cff-47fc-84f1-47c950499bc4 service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:13:02.913509 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f853165a-484c-4c51-b220-8865fda44d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:13:02.916835 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853165a-484c-4c51-b220-8865fda44d44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:02.916835 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853165a-484c-4c51-b220-8865fda44d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:02.916835 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853165a-484c-4c51-b220-8865fda44d44 admin admin] PATH_INFO: `/v3/users/eb20d1b5256545d5a6897acb7461cbda` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:02.916835 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f853165a-484c-4c51-b220-8865fda44d44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb20d1b5256545d5a6897acb7461cbda)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:13:02.917912 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f853165a-484c-4c51-b220-8865fda44d44 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:13:03.050046 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f853165a-484c-4c51-b220-8865fda44d44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb20d1b5256545d5a6897acb7461cbda'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:13:03.050697 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f853165a-484c-4c51-b220-8865fda44d44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb20d1b5256545d5a6897acb7461cbda'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:13:03.104123 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f853165a-484c-4c51-b220-8865fda44d44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb20d1b5256545d5a6897acb7461cbda'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:13:03.104907 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f853165a-484c-4c51-b220-8865fda44d44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb20d1b5256545d5a6897acb7461cbda'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:13:03.117663 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:13:03.119913 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f853165a-484c-4c51-b220-8865fda44d44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb20d1b5256545d5a6897acb7461cbda'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:13:03.139087 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1289/2573] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:13:02 2022] DELETE /identity/v3/users/eb20d1b5256545d5a6897acb7461cbda => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:13:03.489220 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c7f5a-a13d-4793-9816-3572b51de295 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:13:03.490756 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1b9c7f5a-a13d-4793-9816-3572b51de295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:13:03.505421 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c7f5a-a13d-4793-9816-3572b51de295 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:13:03.566275 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0757e4-08c9-49ea-925e-934a97eec096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:13:03.566917 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0757e4-08c9-49ea-925e-934a97eec096 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:03.567200 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0757e4-08c9-49ea-925e-934a97eec096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:03.567476 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0757e4-08c9-49ea-925e-934a97eec096 admin admin] PATH_INFO: `/v3/projects/fe50c2023f4b4bb9908b7c38465fc3d2` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:03.568648 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0757e4-08c9-49ea-925e-934a97eec096 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe50c2023f4b4bb9908b7c38465fc3d2)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:13:03.570566 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0757e4-08c9-49ea-925e-934a97eec096 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:13:03.687100 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-ea0757e4-08c9-49ea-925e-934a97eec096 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe50c2023f4b4bb9908b7c38465fc3d2', 'request_id': 'req-ea0757e4-08c9-49ea-925e-934a97eec096'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:13:03.732357 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-ea0757e4-08c9-49ea-925e-934a97eec096 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe50c2023f4b4bb9908b7c38465fc3d2', 'request_id': 'req-ea0757e4-08c9-49ea-925e-934a97eec096'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:13:03.738276 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-ea0757e4-08c9-49ea-925e-934a97eec096 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe50c2023f4b4bb9908b7c38465fc3d2', 'request_id': 'req-ea0757e4-08c9-49ea-925e-934a97eec096'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:13:03.749916 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-ea0757e4-08c9-49ea-925e-934a97eec096 admin admin] The token cache is being invalidate because project fe50c2023f4b4bb9908b7c38465fc3d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:13:03.750207 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-ea0757e4-08c9-49ea-925e-934a97eec096 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:13:03.750839 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-ea0757e4-08c9-49ea-925e-934a97eec096 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:13:03.752317 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1285/2574] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:13:03 2022] DELETE /identity/v3/projects/fe50c2023f4b4bb9908b7c38465fc3d2 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:13:03.757481 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113e882-260a-4780-9ef4-27780afae5f7 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:03.757766 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113e882-260a-4780-9ef4-27780afae5f7 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:03.758040 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113e882-260a-4780-9ef4-27780afae5f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:03.778607 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-4113e882-260a-4780-9ef4-27780afae5f7 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:13:03.779959 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-4113e882-260a-4780-9ef4-27780afae5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:13:03.839475 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1290/2575] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:13:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:13:03.847243 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dedb1d-29fd-41c9-8877-2879837975e8 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:03.847531 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dedb1d-29fd-41c9-8877-2879837975e8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:03.847836 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dedb1d-29fd-41c9-8877-2879837975e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:03.868162 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-73dedb1d-29fd-41c9-8877-2879837975e8 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:13:03.869405 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-73dedb1d-29fd-41c9-8877-2879837975e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:13:03.878250 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1286/2576] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:13:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:13:03.882293 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4113e882-260a-4780-9ef4-27780afae5f7 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:13:03.892542 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4113e882-260a-4780-9ef4-27780afae5f7 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:13:03.897026 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef87590-c551-4e16-921e-fac2bdd2ba19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:13:03.897615 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef87590-c551-4e16-921e-fac2bdd2ba19 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:03.897892 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef87590-c551-4e16-921e-fac2bdd2ba19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:03.898163 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef87590-c551-4e16-921e-fac2bdd2ba19 admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:03.898797 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef87590-c551-4e16-921e-fac2bdd2ba19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:13:03.899096 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef87590-c551-4e16-921e-fac2bdd2ba19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:13:03.900709 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef87590-c551-4e16-921e-fac2bdd2ba19 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:13:03.912212 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1291/2577] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:13:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:13:03.916577 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dedb1d-29fd-41c9-8877-2879837975e8 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:13:03.926636 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dedb1d-29fd-41c9-8877-2879837975e8 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:13:03.931175 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b17442e-4366-453f-bf05-d81483fe8218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:13:03.931761 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b17442e-4366-453f-bf05-d81483fe8218 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:03.932035 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b17442e-4366-453f-bf05-d81483fe8218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:03.932304 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b17442e-4366-453f-bf05-d81483fe8218 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:03.932956 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b17442e-4366-453f-bf05-d81483fe8218 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:13:03.934830 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b17442e-4366-453f-bf05-d81483fe8218 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:13:04.042547 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1287/2578] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:13:03 2022] POST /identity/v3/projects => generated 369 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:13:04.047715 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef87590-c551-4e16-921e-fac2bdd2ba19 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:13:04.057697 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef87590-c551-4e16-921e-fac2bdd2ba19 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:13:04.117122 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275cea11-3f40-48d6-8831-627f133507e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:13:04.118190 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275cea11-3f40-48d6-8831-627f133507e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:04.118190 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275cea11-3f40-48d6-8831-627f133507e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:04.118374 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275cea11-3f40-48d6-8831-627f133507e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:04.118902 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275cea11-3f40-48d6-8831-627f133507e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:13:04.120856 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275cea11-3f40-48d6-8831-627f133507e4 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:13:04.207322 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1292/2579] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:13:04 2022] POST /identity/v3/users => generated 285 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:13:04.211425 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b17442e-4366-453f-bf05-d81483fe8218 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:13:04.222935 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b17442e-4366-453f-bf05-d81483fe8218 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:13:04.228135 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91832c57-7e8e-4b7f-b48a-7154930fbc4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:13:04.228695 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91832c57-7e8e-4b7f-b48a-7154930fbc4a admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:04.228998 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91832c57-7e8e-4b7f-b48a-7154930fbc4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:04.229268 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91832c57-7e8e-4b7f-b48a-7154930fbc4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:04.229917 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91832c57-7e8e-4b7f-b48a-7154930fbc4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:13:04.231838 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91832c57-7e8e-4b7f-b48a-7154930fbc4a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:13:04.242703 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1288/2580] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:13:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:13:04.246739 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275cea11-3f40-48d6-8831-627f133507e4 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:13:04.256741 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275cea11-3f40-48d6-8831-627f133507e4 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:13:04.261920 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e790cdb-38c0-490d-8dfe-9caad00ba326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:13:04.262551 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e790cdb-38c0-490d-8dfe-9caad00ba326 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:04.262828 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e790cdb-38c0-490d-8dfe-9caad00ba326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:04.263097 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e790cdb-38c0-490d-8dfe-9caad00ba326 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:04.263764 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e790cdb-38c0-490d-8dfe-9caad00ba326 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:13:04.265428 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e790cdb-38c0-490d-8dfe-9caad00ba326 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:13:04.276676 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1293/2581] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:13:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:13:04.283555 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91832c57-7e8e-4b7f-b48a-7154930fbc4a admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:13:04.296539 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91832c57-7e8e-4b7f-b48a-7154930fbc4a admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:13:04.301269 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bcb156-e681-4a3d-a4a7-731237c78205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:13:04.302050 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcb156-e681-4a3d-a4a7-731237c78205 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:04.302478 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcb156-e681-4a3d-a4a7-731237c78205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:04.302597 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcb156-e681-4a3d-a4a7-731237c78205 admin admin] PATH_INFO: `/v3/projects/942bf1d7901d40c3a5cc93adb51d7c51/users/2cdc439e1982440e8c21b97f06f3b9e0/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:04.317568 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bcb156-e681-4a3d-a4a7-731237c78205 admin admin] RBAC: Authorizing `identity:create_grant(project_id=942bf1d7901d40c3a5cc93adb51d7c51, user_id=2cdc439e1982440e8c21b97f06f3b9e0, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:13:04.320239 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bcb156-e681-4a3d-a4a7-731237c78205 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:13:04.412557 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1289/2582] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:13:04 2022] PUT /identity/v3/projects/942bf1d7901d40c3a5cc93adb51d7c51/users/2cdc439e1982440e8c21b97f06f3b9e0/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:13:04.424640 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513b804-5da5-4655-a607-0a3876edfcb1 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:04.424931 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513b804-5da5-4655-a607-0a3876edfcb1 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:04.425209 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513b804-5da5-4655-a607-0a3876edfcb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:04.457187 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-a513b804-5da5-4655-a607-0a3876edfcb1 None None] MFA Rules not processed for user `2cdc439e1982440e8c21b97f06f3b9e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:13:04.458298 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a513b804-5da5-4655-a607-0a3876edfcb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:13:04.503311 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:13:04.528483 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1294/2583] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:13:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:13:04.540227 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bcb156-e681-4a3d-a4a7-731237c78205 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:13:04.542053 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-13bcb156-e681-4a3d-a4a7-731237c78205 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:13:04.553272 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:13:04.611602 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bcb156-e681-4a3d-a4a7-731237c78205 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:13:04.617182 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9745d9-fe8a-429d-84c0-45acf09c0892 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1179503bdab547c99175a27b7ce27b9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:13:04.617824 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9745d9-fe8a-429d-84c0-45acf09c0892 service glance] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:04.618113 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9745d9-fe8a-429d-84c0-45acf09c0892 service glance] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:04.618392 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9745d9-fe8a-429d-84c0-45acf09c0892 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:04.630668 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9745d9-fe8a-429d-84c0-45acf09c0892 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:13:04.632338 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9745d9-fe8a-429d-84c0-45acf09c0892 service glance] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:13:04.642951 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1290/2584] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:13:04 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 01:13:14.607990 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=9 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:13:14.669534 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=23 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:13:26.086408 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513b804-5da5-4655-a607-0a3876edfcb1 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:13:26.148389 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513b804-5da5-4655-a607-0a3876edfcb1 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:13:26.152984 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885e6cc-1550-415a-8ade-7abf37e86798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:13:26.153645 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885e6cc-1550-415a-8ade-7abf37e86798 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:26.153930 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885e6cc-1550-415a-8ade-7abf37e86798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:26.154236 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885e6cc-1550-415a-8ade-7abf37e86798 admin admin] PATH_INFO: `/v3/users/2cdc439e1982440e8c21b97f06f3b9e0` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:26.155297 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2885e6cc-1550-415a-8ade-7abf37e86798 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cdc439e1982440e8c21b97f06f3b9e0)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:13:26.157341 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2885e6cc-1550-415a-8ade-7abf37e86798 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:13:26.317669 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2885e6cc-1550-415a-8ade-7abf37e86798 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cdc439e1982440e8c21b97f06f3b9e0'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:13:26.318320 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2885e6cc-1550-415a-8ade-7abf37e86798 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cdc439e1982440e8c21b97f06f3b9e0'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:13:26.378132 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2885e6cc-1550-415a-8ade-7abf37e86798 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cdc439e1982440e8c21b97f06f3b9e0'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:13:26.378783 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2885e6cc-1550-415a-8ade-7abf37e86798 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cdc439e1982440e8c21b97f06f3b9e0'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:13:26.392086 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2885e6cc-1550-415a-8ade-7abf37e86798 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cdc439e1982440e8c21b97f06f3b9e0'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:13:26.404506 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1295/2585] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:13:26 2022] DELETE /identity/v3/users/2cdc439e1982440e8c21b97f06f3b9e0 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:13:26.408969 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a674c9-ca3d-4506-a771-5f708307e596 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:26.417467 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a674c9-ca3d-4506-a771-5f708307e596 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:26.417467 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a674c9-ca3d-4506-a771-5f708307e596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:26.429040 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-83a674c9-ca3d-4506-a771-5f708307e596 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:13:26.430240 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-83a674c9-ca3d-4506-a771-5f708307e596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:13:26.487851 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1291/2586] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:13:26.497430 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885e6cc-1550-415a-8ade-7abf37e86798 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:13:26.498812 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2885e6cc-1550-415a-8ade-7abf37e86798 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:13:26.557212 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885e6cc-1550-415a-8ade-7abf37e86798 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:13:26.560876 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9334777-e9c2-491e-b468-edeb4b6548f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:13:26.561475 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9334777-e9c2-491e-b468-edeb4b6548f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:26.561755 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9334777-e9c2-491e-b468-edeb4b6548f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:26.562034 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9334777-e9c2-491e-b468-edeb4b6548f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:26.571410 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9334777-e9c2-491e-b468-edeb4b6548f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:13:26.572890 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9334777-e9c2-491e-b468-edeb4b6548f4 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:13:26.583124 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1296/2587] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:13:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:13:26.928901 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a674c9-ca3d-4506-a771-5f708307e596 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:13:26.935017 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-83a674c9-ca3d-4506-a771-5f708307e596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:13:26.943211 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a674c9-ca3d-4506-a771-5f708307e596 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:13:26.947954 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60004802-2134-4da9-8f6e-169990719bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:13:26.948637 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60004802-2134-4da9-8f6e-169990719bd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:26.948922 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60004802-2134-4da9-8f6e-169990719bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:26.949219 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60004802-2134-4da9-8f6e-169990719bd5 admin admin] PATH_INFO: `/v3/projects/942bf1d7901d40c3a5cc93adb51d7c51` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:26.950384 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60004802-2134-4da9-8f6e-169990719bd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=942bf1d7901d40c3a5cc93adb51d7c51)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:13:26.952304 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60004802-2134-4da9-8f6e-169990719bd5 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:13:27.059143 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-60004802-2134-4da9-8f6e-169990719bd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '942bf1d7901d40c3a5cc93adb51d7c51', 'request_id': 'req-60004802-2134-4da9-8f6e-169990719bd5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:13:27.118758 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-60004802-2134-4da9-8f6e-169990719bd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '942bf1d7901d40c3a5cc93adb51d7c51', 'request_id': 'req-60004802-2134-4da9-8f6e-169990719bd5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:13:27.121716 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-60004802-2134-4da9-8f6e-169990719bd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '942bf1d7901d40c3a5cc93adb51d7c51', 'request_id': 'req-60004802-2134-4da9-8f6e-169990719bd5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:13:27.134017 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-60004802-2134-4da9-8f6e-169990719bd5 admin admin] The token cache is being invalidate because project 942bf1d7901d40c3a5cc93adb51d7c51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:13:27.134316 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-60004802-2134-4da9-8f6e-169990719bd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:13:27.135174 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-60004802-2134-4da9-8f6e-169990719bd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:13:27.136742 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1292/2588] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:13:26 2022] DELETE /identity/v3/projects/942bf1d7901d40c3a5cc93adb51d7c51 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:13:27.141726 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511a29f-f6f1-440b-80d8-09c0f0efa638 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:27.142241 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511a29f-f6f1-440b-80d8-09c0f0efa638 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:27.142359 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511a29f-f6f1-440b-80d8-09c0f0efa638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:27.162020 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-f511a29f-f6f1-440b-80d8-09c0f0efa638 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:13:27.163231 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f511a29f-f6f1-440b-80d8-09c0f0efa638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:13:27.180674 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:13:27.228954 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1297/2589] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:13:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:13:27.237062 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29f260-5157-405a-8062-e4b08d9f7c83 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:27.237866 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29f260-5157-405a-8062-e4b08d9f7c83 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:27.238145 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29f260-5157-405a-8062-e4b08d9f7c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:27.259154 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5b29f260-5157-405a-8062-e4b08d9f7c83 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:13:27.261107 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5b29f260-5157-405a-8062-e4b08d9f7c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:13:27.269084 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1293/2590] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:13:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:13:27.273529 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511a29f-f6f1-440b-80d8-09c0f0efa638 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:13:27.283871 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511a29f-f6f1-440b-80d8-09c0f0efa638 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:13:27.288695 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1addd811-eab3-4ded-82b6-12cd04d6b785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:13:27.289295 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1addd811-eab3-4ded-82b6-12cd04d6b785 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:27.289575 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1addd811-eab3-4ded-82b6-12cd04d6b785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:27.289848 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1addd811-eab3-4ded-82b6-12cd04d6b785 admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:27.290488 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1addd811-eab3-4ded-82b6-12cd04d6b785 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:13:27.290820 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1addd811-eab3-4ded-82b6-12cd04d6b785 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:13:27.292412 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1addd811-eab3-4ded-82b6-12cd04d6b785 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:13:27.304257 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1298/2591] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:13:27.313186 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29f260-5157-405a-8062-e4b08d9f7c83 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:13:27.318771 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29f260-5157-405a-8062-e4b08d9f7c83 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:13:27.323507 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00987251-4ebd-454e-ae69-6b0a29300023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:13:27.324105 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00987251-4ebd-454e-ae69-6b0a29300023 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:27.324390 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00987251-4ebd-454e-ae69-6b0a29300023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:27.324699 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00987251-4ebd-454e-ae69-6b0a29300023 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:27.325333 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00987251-4ebd-454e-ae69-6b0a29300023 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:13:27.327227 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00987251-4ebd-454e-ae69-6b0a29300023 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:13:27.442393 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1294/2592] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:13:27 2022] POST /identity/v3/projects => generated 371 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:13:27.446595 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1addd811-eab3-4ded-82b6-12cd04d6b785 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:13:27.457009 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1addd811-eab3-4ded-82b6-12cd04d6b785 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:13:27.527489 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a0994-a9e5-4f50-ae65-eaf5fdf9be5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:13:27.528091 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a0994-a9e5-4f50-ae65-eaf5fdf9be5a admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:27.528411 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a0994-a9e5-4f50-ae65-eaf5fdf9be5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:27.528699 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a0994-a9e5-4f50-ae65-eaf5fdf9be5a admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:27.529351 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730a0994-a9e5-4f50-ae65-eaf5fdf9be5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:13:27.531405 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730a0994-a9e5-4f50-ae65-eaf5fdf9be5a admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:13:27.607005 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1299/2593] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:13:27 2022] POST /identity/v3/users => generated 286 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:13:27.611265 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00987251-4ebd-454e-ae69-6b0a29300023 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:13:27.622455 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00987251-4ebd-454e-ae69-6b0a29300023 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:13:27.627327 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ddb98c-8fe9-47c9-aa24-5af4d95d74c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:13:27.627909 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ddb98c-8fe9-47c9-aa24-5af4d95d74c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:27.632096 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ddb98c-8fe9-47c9-aa24-5af4d95d74c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:27.632096 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ddb98c-8fe9-47c9-aa24-5af4d95d74c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:27.632096 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ddb98c-8fe9-47c9-aa24-5af4d95d74c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:13:27.632096 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ddb98c-8fe9-47c9-aa24-5af4d95d74c5 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:13:27.646128 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1295/2594] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:13:27.647372 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a0994-a9e5-4f50-ae65-eaf5fdf9be5a admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:13:27.657701 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a0994-a9e5-4f50-ae65-eaf5fdf9be5a admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:13:27.663247 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e396afc-60bd-4ca1-be8d-df951c5ec185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:13:27.663852 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e396afc-60bd-4ca1-be8d-df951c5ec185 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:27.664139 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e396afc-60bd-4ca1-be8d-df951c5ec185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:27.664419 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e396afc-60bd-4ca1-be8d-df951c5ec185 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:27.665105 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e396afc-60bd-4ca1-be8d-df951c5ec185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:13:27.666701 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e396afc-60bd-4ca1-be8d-df951c5ec185 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:13:27.680148 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1300/2595] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:13:27.683778 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ddb98c-8fe9-47c9-aa24-5af4d95d74c5 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:13:27.694013 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ddb98c-8fe9-47c9-aa24-5af4d95d74c5 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:13:27.701411 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f779d153-26c1-43aa-91be-e74196a00adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:13:27.701411 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779d153-26c1-43aa-91be-e74196a00adc admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:27.701411 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779d153-26c1-43aa-91be-e74196a00adc admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:27.701411 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779d153-26c1-43aa-91be-e74196a00adc admin admin] PATH_INFO: `/v3/projects/2fe55a7032cb47ad98b86885aad23d6c/users/48b714ce386c44c88914f5a74884a52a/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:27.716859 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f779d153-26c1-43aa-91be-e74196a00adc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fe55a7032cb47ad98b86885aad23d6c, user_id=48b714ce386c44c88914f5a74884a52a, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:13:27.719663 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f779d153-26c1-43aa-91be-e74196a00adc admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:13:27.824480 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1296/2596] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:13:27 2022] PUT /identity/v3/projects/2fe55a7032cb47ad98b86885aad23d6c/users/48b714ce386c44c88914f5a74884a52a/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:13:27.829561 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243a4f2-438d-4234-a152-159f78ae55ca None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:27.829851 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243a4f2-438d-4234-a152-159f78ae55ca None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:27.830131 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243a4f2-438d-4234-a152-159f78ae55ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:27.849653 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-6243a4f2-438d-4234-a152-159f78ae55ca None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:13:27.850941 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-6243a4f2-438d-4234-a152-159f78ae55ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:13:27.910546 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1301/2597] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:13:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:13:27.920166 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f779d153-26c1-43aa-91be-e74196a00adc admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:13:27.921854 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f779d153-26c1-43aa-91be-e74196a00adc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:13:27.944858 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:13:27.981094 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f779d153-26c1-43aa-91be-e74196a00adc admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:13:27.984695 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e30183-84ea-46d3-ae35-6320d678c963 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:13:27.985295 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e30183-84ea-46d3-ae35-6320d678c963 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:27.985573 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e30183-84ea-46d3-ae35-6320d678c963 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:27.985843 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e30183-84ea-46d3-ae35-6320d678c963 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:27.995095 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e30183-84ea-46d3-ae35-6320d678c963 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:13:27.996537 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e30183-84ea-46d3-ae35-6320d678c963 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:13:28.006476 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1297/2598] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:13:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 01:13:29.643290 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1550f20-8222-4d5c-affd-d0b1025e6f0a None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:29.644146 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1550f20-8222-4d5c-affd-d0b1025e6f0a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:29.644440 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1550f20-8222-4d5c-affd-d0b1025e6f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:29.690107 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-a1550f20-8222-4d5c-affd-d0b1025e6f0a None None] MFA Rules not processed for user `48b714ce386c44c88914f5a74884a52a`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:13:29.691296 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a1550f20-8222-4d5c-affd-d0b1025e6f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:13:29.713842 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:13:29.764744 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1302/2599] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:13:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:13:29.776733 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e30183-84ea-46d3-ae35-6320d678c963 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:13:29.778357 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-47e30183-84ea-46d3-ae35-6320d678c963 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:13:29.838003 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e30183-84ea-46d3-ae35-6320d678c963 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:13:29.841582 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084217a-7871-410c-87ee-4f8af16c0d17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:13:29.842172 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084217a-7871-410c-87ee-4f8af16c0d17 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:29.842449 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084217a-7871-410c-87ee-4f8af16c0d17 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:29.842719 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084217a-7871-410c-87ee-4f8af16c0d17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:29.851625 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9084217a-7871-410c-87ee-4f8af16c0d17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:13:29.853432 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9084217a-7871-410c-87ee-4f8af16c0d17 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:13:29.861810 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1298/2600] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:13:29 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:13:31.518981 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472c493-f3af-4cf1-9573-c80e1e61f9ca None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:31.519516 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472c493-f3af-4cf1-9573-c80e1e61f9ca None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:31.519716 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472c493-f3af-4cf1-9573-c80e1e61f9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:31.538935 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-1472c493-f3af-4cf1-9573-c80e1e61f9ca None None] MFA Rules not processed for user `48b714ce386c44c88914f5a74884a52a`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:13:31.540143 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-1472c493-f3af-4cf1-9573-c80e1e61f9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:13:31.547053 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1303/2601] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:13:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:13:31.788898 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084217a-7871-410c-87ee-4f8af16c0d17 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:13:31.800920 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084217a-7871-410c-87ee-4f8af16c0d17 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:13:31.803620 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96165be-1c95-44b9-a893-ef47a72b275c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:13:31.804231 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96165be-1c95-44b9-a893-ef47a72b275c service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:31.804546 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96165be-1c95-44b9-a893-ef47a72b275c service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:31.804824 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96165be-1c95-44b9-a893-ef47a72b275c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:31.806634 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e96165be-1c95-44b9-a893-ef47a72b275c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:13:31.817502 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96165be-1c95-44b9-a893-ef47a72b275c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:13:31.818929 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96165be-1c95-44b9-a893-ef47a72b275c service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:13:31.827128 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1299/2602] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:13:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:13:41.533235 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=22 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:13:41.823394 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=49 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:13:45.186575 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1b3fd-6af0-4aaa-91db-0c075cde6edb None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:45.190173 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1b3fd-6af0-4aaa-91db-0c075cde6edb None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:45.190173 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1b3fd-6af0-4aaa-91db-0c075cde6edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:45.207344 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-52c1b3fd-6af0-4aaa-91db-0c075cde6edb None None] MFA Rules not processed for user `0c5e9731f6204b44841b8524762d5a77`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:13:45.208511 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-52c1b3fd-6af0-4aaa-91db-0c075cde6edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:13:45.267598 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1304/2603] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:13:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:13:55.046307 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814f84e-f03e-47a8-bf94-4bd4c78d758a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:13:55.046773 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814f84e-f03e-47a8-bf94-4bd4c78d758a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:13:55.046877 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814f84e-f03e-47a8-bf94-4bd4c78d758a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:13:55.067524 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c814f84e-f03e-47a8-bf94-4bd4c78d758a None None] MFA Rules not processed for user `48b714ce386c44c88914f5a74884a52a`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:13:55.068820 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c814f84e-f03e-47a8-bf94-4bd4c78d758a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:13:55.076062 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1300/2604] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:13:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:13:55.260851 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=25 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:14:05.061917 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:14:20.235850 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b82d7-250a-4265-9b6f-40b0d5b5ebac None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:14:20.236326 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b82d7-250a-4265-9b6f-40b0d5b5ebac None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:14:20.236420 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b82d7-250a-4265-9b6f-40b0d5b5ebac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:14:20.257107 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-325b82d7-250a-4265-9b6f-40b0d5b5ebac None None] MFA Rules not processed for user `d6d01ca673a1404c9b4cea51036c595c`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:14:20.258312 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-325b82d7-250a-4265-9b6f-40b0d5b5ebac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:14:20.315625 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1305/2605] 10.222.0.27 () {58 vars in 1009 bytes} [Sun Sep 4 01:14:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:14:20.461440 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814f84e-f03e-47a8-bf94-4bd4c78d758a None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:14:20.462844 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c814f84e-f03e-47a8-bf94-4bd4c78d758a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:14:20.531932 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814f84e-f03e-47a8-bf94-4bd4c78d758a None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:14:20.536916 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499300a-498b-4a44-ae8c-2bd8e70bbd71 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2297d8b3fe8e47eb9fd39a6e3eea8f54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:14:20.537516 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499300a-498b-4a44-ae8c-2bd8e70bbd71 service swift] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:14:20.537796 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499300a-498b-4a44-ae8c-2bd8e70bbd71 service swift] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:14:20.538069 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499300a-498b-4a44-ae8c-2bd8e70bbd71 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:14:20.546592 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c499300a-498b-4a44-ae8c-2bd8e70bbd71 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:14:20.548150 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c499300a-498b-4a44-ae8c-2bd8e70bbd71 service swift] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:14:20.555923 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1301/2606] 10.222.0.27 () {60 vars in 1384 bytes} [Sun Sep 4 01:14:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 04 01:14:29.503939 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68019875-6457-457d-bf0f-03bf7f49e9bc None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:14:29.504480 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68019875-6457-457d-bf0f-03bf7f49e9bc None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:14:29.504480 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68019875-6457-457d-bf0f-03bf7f49e9bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:14:29.548863 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-68019875-6457-457d-bf0f-03bf7f49e9bc None None] MFA Rules not processed for user `d6d01ca673a1404c9b4cea51036c595c`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:14:29.550231 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-68019875-6457-457d-bf0f-03bf7f49e9bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:14:29.589969 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1306/2607] 10.222.0.27 () {58 vars in 1009 bytes} [Sun Sep 4 01:14:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:14:29.608774 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499300a-498b-4a44-ae8c-2bd8e70bbd71 service swift] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:14:29.663944 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499300a-498b-4a44-ae8c-2bd8e70bbd71 service swift] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:14:29.672048 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6cbcb2-1405-41e3-9a5e-9a2182a78ac1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2297d8b3fe8e47eb9fd39a6e3eea8f54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:14:29.672953 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6cbcb2-1405-41e3-9a5e-9a2182a78ac1 service swift] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:14:29.673380 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6cbcb2-1405-41e3-9a5e-9a2182a78ac1 service swift] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:14:29.673665 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6cbcb2-1405-41e3-9a5e-9a2182a78ac1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:14:29.684843 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6cbcb2-1405-41e3-9a5e-9a2182a78ac1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:14:29.687954 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6cbcb2-1405-41e3-9a5e-9a2182a78ac1 service swift] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:14:29.699860 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1302/2608] 10.222.0.27 () {60 vars in 1384 bytes} [Sun Sep 4 01:14:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 04 01:14:37.864175 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842186b-a042-49e1-9bd9-10c876c1f364 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:14:37.865068 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842186b-a042-49e1-9bd9-10c876c1f364 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:14:37.865352 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842186b-a042-49e1-9bd9-10c876c1f364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:14:37.888414 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-7842186b-a042-49e1-9bd9-10c876c1f364 None None] MFA Rules not processed for user `d6d01ca673a1404c9b4cea51036c595c`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:14:37.889736 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-7842186b-a042-49e1-9bd9-10c876c1f364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:14:37.903503 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1307/2609] 10.222.0.27 () {58 vars in 967 bytes} [Sun Sep 4 01:14:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:14:37.924354 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6cbcb2-1405-41e3-9a5e-9a2182a78ac1 service swift] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:14:37.936649 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6cbcb2-1405-41e3-9a5e-9a2182a78ac1 service swift] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:14:37.943122 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e896f-5ef8-41ee-a6b3-5c173a1fb3cd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2297d8b3fe8e47eb9fd39a6e3eea8f54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:14:37.943969 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e896f-5ef8-41ee-a6b3-5c173a1fb3cd service swift] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:14:37.944496 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e896f-5ef8-41ee-a6b3-5c173a1fb3cd service swift] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:14:37.944936 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e896f-5ef8-41ee-a6b3-5c173a1fb3cd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:14:37.956012 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e896f-5ef8-41ee-a6b3-5c173a1fb3cd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:14:37.957708 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e896f-5ef8-41ee-a6b3-5c173a1fb3cd service swift] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:14:37.970762 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1303/2610] 10.222.0.27 () {60 vars in 1384 bytes} [Sun Sep 4 01:14:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 04 01:14:41.002235 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7842186b-a042-49e1-9bd9-10c876c1f364 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:14:41.039338 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7842186b-a042-49e1-9bd9-10c876c1f364 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:14:41.044686 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037ead07-9dee-4789-a9d8-ec8b0b8d9ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:14:41.045425 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ead07-9dee-4789-a9d8-ec8b0b8d9ecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:14:41.045773 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ead07-9dee-4789-a9d8-ec8b0b8d9ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:14:41.046246 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ead07-9dee-4789-a9d8-ec8b0b8d9ecc admin admin] PATH_INFO: `/v3/users/48b714ce386c44c88914f5a74884a52a` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:14:41.047587 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037ead07-9dee-4789-a9d8-ec8b0b8d9ecc admin admin] RBAC: Authorizing `identity:delete_user(user_id=48b714ce386c44c88914f5a74884a52a)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:14:41.049580 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037ead07-9dee-4789-a9d8-ec8b0b8d9ecc admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:14:41.283228 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-037ead07-9dee-4789-a9d8-ec8b0b8d9ecc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48b714ce386c44c88914f5a74884a52a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:14:41.283878 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-037ead07-9dee-4789-a9d8-ec8b0b8d9ecc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48b714ce386c44c88914f5a74884a52a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:14:41.366747 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-037ead07-9dee-4789-a9d8-ec8b0b8d9ecc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48b714ce386c44c88914f5a74884a52a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:14:41.367378 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-037ead07-9dee-4789-a9d8-ec8b0b8d9ecc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48b714ce386c44c88914f5a74884a52a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:14:41.381416 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-037ead07-9dee-4789-a9d8-ec8b0b8d9ecc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48b714ce386c44c88914f5a74884a52a'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:14:41.394941 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1308/2611] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:14:41 2022] DELETE /identity/v3/users/48b714ce386c44c88914f5a74884a52a => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:14:41.637812 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e896f-5ef8-41ee-a6b3-5c173a1fb3cd service swift] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:14:41.648907 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e896f-5ef8-41ee-a6b3-5c173a1fb3cd service swift] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:14:41.707289 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a84d81-43fe-42f1-853d-6fb7a4014469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:14:41.707948 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a84d81-43fe-42f1-853d-6fb7a4014469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:14:41.708234 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a84d81-43fe-42f1-853d-6fb7a4014469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:14:41.708554 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a84d81-43fe-42f1-853d-6fb7a4014469 admin admin] PATH_INFO: `/v3/projects/2fe55a7032cb47ad98b86885aad23d6c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:14:41.709728 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a84d81-43fe-42f1-853d-6fb7a4014469 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fe55a7032cb47ad98b86885aad23d6c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:14:41.711745 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a84d81-43fe-42f1-853d-6fb7a4014469 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:14:41.786079 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:14:41.795571 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-18a84d81-43fe-42f1-853d-6fb7a4014469 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fe55a7032cb47ad98b86885aad23d6c', 'request_id': 'req-18a84d81-43fe-42f1-853d-6fb7a4014469'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:14:41.835381 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-18a84d81-43fe-42f1-853d-6fb7a4014469 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fe55a7032cb47ad98b86885aad23d6c', 'request_id': 'req-18a84d81-43fe-42f1-853d-6fb7a4014469'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:14:41.840976 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-18a84d81-43fe-42f1-853d-6fb7a4014469 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fe55a7032cb47ad98b86885aad23d6c', 'request_id': 'req-18a84d81-43fe-42f1-853d-6fb7a4014469'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:14:41.863789 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-18a84d81-43fe-42f1-853d-6fb7a4014469 admin admin] The token cache is being invalidate because project 2fe55a7032cb47ad98b86885aad23d6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:14:41.864301 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-18a84d81-43fe-42f1-853d-6fb7a4014469 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:14:41.865377 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-18a84d81-43fe-42f1-853d-6fb7a4014469 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:14:41.866882 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1304/2612] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:14:41 2022] DELETE /identity/v3/projects/2fe55a7032cb47ad98b86885aad23d6c => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:14:41.873275 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35619d-eded-49a6-ae19-40bc9a40c577 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:14:41.873566 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35619d-eded-49a6-ae19-40bc9a40c577 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:14:41.873846 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35619d-eded-49a6-ae19-40bc9a40c577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:14:41.879504 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:14:41.933440 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-3f35619d-eded-49a6-ae19-40bc9a40c577 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:14:41.934674 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3f35619d-eded-49a6-ae19-40bc9a40c577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:14:41.997076 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1309/2613] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:14:42.004968 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d2276-f982-4f19-a77a-8de69f76817f None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:14:42.005262 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d2276-f982-4f19-a77a-8de69f76817f None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:14:42.005552 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d2276-f982-4f19-a77a-8de69f76817f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:14:42.026534 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f14d2276-f982-4f19-a77a-8de69f76817f None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:14:42.027763 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f14d2276-f982-4f19-a77a-8de69f76817f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:14:42.035782 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1305/2614] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:14:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:14:42.040234 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f35619d-eded-49a6-ae19-40bc9a40c577 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:14:42.050836 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f35619d-eded-49a6-ae19-40bc9a40c577 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:14:42.055512 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a438127f-d7a5-41a0-8d40-b39b84f4099e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:14:42.056118 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438127f-d7a5-41a0-8d40-b39b84f4099e admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:14:42.056400 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438127f-d7a5-41a0-8d40-b39b84f4099e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:14:42.056854 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438127f-d7a5-41a0-8d40-b39b84f4099e admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:14:42.057825 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a438127f-d7a5-41a0-8d40-b39b84f4099e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:14:42.058132 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a438127f-d7a5-41a0-8d40-b39b84f4099e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:14:42.059732 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a438127f-d7a5-41a0-8d40-b39b84f4099e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:14:42.075150 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1310/2615] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:14:42.076267 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d2276-f982-4f19-a77a-8de69f76817f None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:14:42.085602 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d2276-f982-4f19-a77a-8de69f76817f None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:14:42.091174 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4d568-8ddc-42d9-88e4-6cde63534c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:14:42.091782 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4d568-8ddc-42d9-88e4-6cde63534c6e admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:14:42.092072 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4d568-8ddc-42d9-88e4-6cde63534c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:14:42.092347 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4d568-8ddc-42d9-88e4-6cde63534c6e admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:14:42.093067 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb4d568-8ddc-42d9-88e4-6cde63534c6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:14:42.095061 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb4d568-8ddc-42d9-88e4-6cde63534c6e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:14:42.179618 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1306/2616] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:14:42 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:14:42.184300 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a438127f-d7a5-41a0-8d40-b39b84f4099e admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:14:42.194306 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a438127f-d7a5-41a0-8d40-b39b84f4099e admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:14:42.250864 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac3b47-92b5-4cd1-8165-fe2e7e64176e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:14:42.251475 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac3b47-92b5-4cd1-8165-fe2e7e64176e admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:14:42.251757 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac3b47-92b5-4cd1-8165-fe2e7e64176e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:14:42.252055 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac3b47-92b5-4cd1-8165-fe2e7e64176e admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:14:42.252724 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ac3b47-92b5-4cd1-8165-fe2e7e64176e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:14:42.254699 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ac3b47-92b5-4cd1-8165-fe2e7e64176e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:14:42.339417 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1311/2617] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:14:42 2022] POST /identity/v3/users => generated 288 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:14:42.348687 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4d568-8ddc-42d9-88e4-6cde63534c6e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:14:42.359390 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4d568-8ddc-42d9-88e4-6cde63534c6e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:14:42.363978 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf40d0-db0e-4713-8337-be442f55b394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:14:42.364608 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf40d0-db0e-4713-8337-be442f55b394 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:14:42.364890 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf40d0-db0e-4713-8337-be442f55b394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:14:42.365164 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf40d0-db0e-4713-8337-be442f55b394 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:14:42.365798 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf40d0-db0e-4713-8337-be442f55b394 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:14:42.367432 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf40d0-db0e-4713-8337-be442f55b394 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:14:42.377671 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1307/2618] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:14:42.381680 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac3b47-92b5-4cd1-8165-fe2e7e64176e admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:14:42.392330 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac3b47-92b5-4cd1-8165-fe2e7e64176e admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:14:42.397414 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3626cb-56f8-49be-bc3d-208f43228a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:14:42.398075 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3626cb-56f8-49be-bc3d-208f43228a45 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:14:42.398361 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3626cb-56f8-49be-bc3d-208f43228a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:14:42.398636 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3626cb-56f8-49be-bc3d-208f43228a45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:14:42.399274 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3626cb-56f8-49be-bc3d-208f43228a45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:14:42.401620 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3626cb-56f8-49be-bc3d-208f43228a45 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:14:42.412352 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1312/2619] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:14:42.416911 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf40d0-db0e-4713-8337-be442f55b394 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:14:42.426708 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf40d0-db0e-4713-8337-be442f55b394 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:14:42.431429 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae753f1-ab47-4abb-84a0-87f30e3438d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:14:42.432743 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae753f1-ab47-4abb-84a0-87f30e3438d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:14:42.432934 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae753f1-ab47-4abb-84a0-87f30e3438d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:14:42.433207 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae753f1-ab47-4abb-84a0-87f30e3438d9 admin admin] PATH_INFO: `/v3/projects/db1b33cc7cd2466b9f9cf4e0a631941e/users/2c9fe988cac442bfa1549d9b183498c5/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:14:42.447532 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae753f1-ab47-4abb-84a0-87f30e3438d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db1b33cc7cd2466b9f9cf4e0a631941e, user_id=2c9fe988cac442bfa1549d9b183498c5, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:14:42.450186 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae753f1-ab47-4abb-84a0-87f30e3438d9 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:14:42.528353 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1308/2620] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:14:42 2022] PUT /identity/v3/projects/db1b33cc7cd2466b9f9cf4e0a631941e/users/2c9fe988cac442bfa1549d9b183498c5/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:14:42.538319 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e89cd-1a27-48d5-88bd-7db11c0637a8 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:14:42.538602 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e89cd-1a27-48d5-88bd-7db11c0637a8 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:14:42.538875 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e89cd-1a27-48d5-88bd-7db11c0637a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:14:42.567831 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-0b0e89cd-1a27-48d5-88bd-7db11c0637a8 None None] MFA Rules not processed for user `2c9fe988cac442bfa1549d9b183498c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:14:42.569094 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-0b0e89cd-1a27-48d5-88bd-7db11c0637a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:14:42.598452 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:14:42.635917 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1313/2621] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:14:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:14:42.647006 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae753f1-ab47-4abb-84a0-87f30e3438d9 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:14:42.648813 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7ae753f1-ab47-4abb-84a0-87f30e3438d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:14:42.709977 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae753f1-ab47-4abb-84a0-87f30e3438d9 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:14:42.714211 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef342929-d3ed-4d96-b664-e67002e065f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:14:42.714823 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef342929-d3ed-4d96-b664-e67002e065f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:14:42.715104 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef342929-d3ed-4d96-b664-e67002e065f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:14:42.715377 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef342929-d3ed-4d96-b664-e67002e065f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:14:42.723885 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef342929-d3ed-4d96-b664-e67002e065f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:14:42.725283 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef342929-d3ed-4d96-b664-e67002e065f5 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:14:42.734923 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1309/2622] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:14:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:14:44.432235 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e5833-d974-4905-9d35-fd5ea976db43 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:14:44.432989 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e5833-d974-4905-9d35-fd5ea976db43 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:14:44.432989 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e5833-d974-4905-9d35-fd5ea976db43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:14:44.452316 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-070e5833-d974-4905-9d35-fd5ea976db43 None None] MFA Rules not processed for user `2c9fe988cac442bfa1549d9b183498c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:14:44.453574 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-070e5833-d974-4905-9d35-fd5ea976db43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:14:44.461071 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1314/2623] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:14:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:14:44.769211 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef342929-d3ed-4d96-b664-e67002e065f5 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:14:44.770663 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ef342929-d3ed-4d96-b664-e67002e065f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:14:44.830639 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:14:44.868369 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef342929-d3ed-4d96-b664-e67002e065f5 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:14:44.874325 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7829c692-b11a-4e62-9683-f341df22b69f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '018802a8d9c041119ad708430f885a89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:14:44.874924 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829c692-b11a-4e62-9683-f341df22b69f service placement] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:14:44.875203 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829c692-b11a-4e62-9683-f341df22b69f service placement] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:14:44.875475 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829c692-b11a-4e62-9683-f341df22b69f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:14:44.877193 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7829c692-b11a-4e62-9683-f341df22b69f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:14:44.889538 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7829c692-b11a-4e62-9683-f341df22b69f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:14:44.891084 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7829c692-b11a-4e62-9683-f341df22b69f service placement] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:14:44.900929 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1310/2624] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:14:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:14:47.706715 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4880e41-cba4-4b4e-87aa-66e13a8778d1 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:14:47.707292 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4880e41-cba4-4b4e-87aa-66e13a8778d1 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:14:47.707292 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4880e41-cba4-4b4e-87aa-66e13a8778d1 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:14:47.708844 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1315/2625] 10.222.0.27 () {54 vars in 921 bytes} [Sun Sep 4 01:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:14:47.714940 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4ceb8-e0aa-4aef-9d52-71a30cb0227c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:14:47.715429 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4ceb8-e0aa-4aef-9d52-71a30cb0227c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:14:47.715547 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4ceb8-e0aa-4aef-9d52-71a30cb0227c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:14:47.748610 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-2fc4ceb8-e0aa-4aef-9d52-71a30cb0227c None None] MFA Rules not processed for user `68bc00bda5194cd8a7928f8c4df4ccd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:14:47.749823 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2fc4ceb8-e0aa-4aef-9d52-71a30cb0227c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:14:47.831903 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1311/2626] 10.222.0.27 () {58 vars in 1018 bytes} [Sun Sep 4 01:14:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:14:47.861070 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4880e41-cba4-4b4e-87aa-66e13a8778d1 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:14:47.863102 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e4880e41-cba4-4b4e-87aa-66e13a8778d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:14:47.924161 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4880e41-cba4-4b4e-87aa-66e13a8778d1 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:14:47.927842 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321e6ab-3328-4bf5-a7f3-0f4ead139f51 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1179503bdab547c99175a27b7ce27b9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:14:47.928508 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321e6ab-3328-4bf5-a7f3-0f4ead139f51 service glance] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:14:47.928795 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321e6ab-3328-4bf5-a7f3-0f4ead139f51 service glance] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:14:47.929067 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321e6ab-3328-4bf5-a7f3-0f4ead139f51 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:14:47.939507 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f321e6ab-3328-4bf5-a7f3-0f4ead139f51 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:14:47.941042 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f321e6ab-3328-4bf5-a7f3-0f4ead139f51 service glance] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:14:47.952055 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1316/2627] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:14:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:14:48.158300 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1703b-8b6e-4b6c-a53a-5cc331bee36a None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:14:48.158705 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1703b-8b6e-4b6c-a53a-5cc331bee36a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:14:48.159195 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1703b-8b6e-4b6c-a53a-5cc331bee36a None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:14:48.160750 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1312/2628] 10.222.0.27 () {54 vars in 921 bytes} [Sun Sep 4 01:14:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:14:48.166356 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0eaeaf-3caa-4bb7-8669-08d46c38f883 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:14:48.166648 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0eaeaf-3caa-4bb7-8669-08d46c38f883 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:14:48.166934 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0eaeaf-3caa-4bb7-8669-08d46c38f883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:14:48.201045 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-8c0eaeaf-3caa-4bb7-8669-08d46c38f883 None None] MFA Rules not processed for user `861c278f8bf34b3bb407b2e051119ee9`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:14:48.202281 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-8c0eaeaf-3caa-4bb7-8669-08d46c38f883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:14:48.275336 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1317/2629] 10.222.0.27 () {58 vars in 1018 bytes} [Sun Sep 4 01:14:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:14:48.289013 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1703b-8b6e-4b6c-a53a-5cc331bee36a None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:14:48.291243 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-fbf1703b-8b6e-4b6c-a53a-5cc331bee36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:14:48.305461 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1703b-8b6e-4b6c-a53a-5cc331bee36a None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:14:48.309695 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2402fdd-2259-4b42-b32b-fce5ca841d21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:14:48.310305 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2402fdd-2259-4b42-b32b-fce5ca841d21 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:14:48.310583 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2402fdd-2259-4b42-b32b-fce5ca841d21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:14:48.310853 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2402fdd-2259-4b42-b32b-fce5ca841d21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:14:48.318602 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2402fdd-2259-4b42-b32b-fce5ca841d21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:14:48.320150 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2402fdd-2259-4b42-b32b-fce5ca841d21 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:14:48.330115 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1313/2630] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:14:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:14:48.436592 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e19be-b463-456d-8702-02ec257d13e4 None None] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:14:48.437125 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e19be-b463-456d-8702-02ec257d13e4 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:14:48.437403 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e19be-b463-456d-8702-02ec257d13e4 None None] PATH_INFO: `/` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:14:48.438763 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1318/2631] 10.222.0.27 () {54 vars in 921 bytes} [Sun Sep 4 01:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 04 01:14:48.444596 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf325561-9450-4d37-8fc3-91b6ca0c1082 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:14:48.444909 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf325561-9450-4d37-8fc3-91b6ca0c1082 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:14:48.445498 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf325561-9450-4d37-8fc3-91b6ca0c1082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:14:48.467042 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-cf325561-9450-4d37-8fc3-91b6ca0c1082 None None] MFA Rules not processed for user `68bc00bda5194cd8a7928f8c4df4ccd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:14:48.468275 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-cf325561-9450-4d37-8fc3-91b6ca0c1082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:14:48.488347 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1314/2632] 10.222.0.27 () {58 vars in 1018 bytes} [Sun Sep 4 01:14:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:14:48.506429 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1e19be-b463-456d-8702-02ec257d13e4 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:14:48.523174 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1e19be-b463-456d-8702-02ec257d13e4 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:14:48.528503 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ec792f-f0fb-4742-877a-3faa1f465356 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:14:48.529788 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec792f-f0fb-4742-877a-3faa1f465356 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:14:48.530091 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec792f-f0fb-4742-877a-3faa1f465356 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:14:48.530368 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec792f-f0fb-4742-877a-3faa1f465356 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:14:48.557429 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ec792f-f0fb-4742-877a-3faa1f465356 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:14:48.559082 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ec792f-f0fb-4742-877a-3faa1f465356 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:14:48.570888 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1319/2633] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:14:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:14:53.143841 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad6d9a-4bb1-4c82-8777-a92e1f15d3ef None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:14:53.144317 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad6d9a-4bb1-4c82-8777-a92e1f15d3ef None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:14:53.144414 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad6d9a-4bb1-4c82-8777-a92e1f15d3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:14:53.165311 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f7ad6d9a-4bb1-4c82-8777-a92e1f15d3ef None None] MFA Rules not processed for user `d6d01ca673a1404c9b4cea51036c595c`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:14:53.166519 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f7ad6d9a-4bb1-4c82-8777-a92e1f15d3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:14:53.222908 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1315/2634] 10.222.0.27 () {58 vars in 1009 bytes} [Sun Sep 4 01:14:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:14:53.236064 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ec792f-f0fb-4742-877a-3faa1f465356 service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:14:53.237585 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-25ec792f-f0fb-4742-877a-3faa1f465356 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:14:53.285027 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:14:53.309343 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ec792f-f0fb-4742-877a-3faa1f465356 service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:14:53.314901 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a63fd6-4b8a-4181-b9d9-903bfc2be92f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2297d8b3fe8e47eb9fd39a6e3eea8f54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:14:53.315888 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a63fd6-4b8a-4181-b9d9-903bfc2be92f service swift] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:14:53.316668 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a63fd6-4b8a-4181-b9d9-903bfc2be92f service swift] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:14:53.328673 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a63fd6-4b8a-4181-b9d9-903bfc2be92f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:14:53.328673 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a63fd6-4b8a-4181-b9d9-903bfc2be92f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:14:53.328673 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a63fd6-4b8a-4181-b9d9-903bfc2be92f service swift] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:14:53.335788 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1320/2635] 10.222.0.27 () {60 vars in 1384 bytes} [Sun Sep 4 01:14:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 04 01:15:03.216044 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:15:03.332392 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=13 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:15:03.933127 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad6d9a-4bb1-4c82-8777-a92e1f15d3ef None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:15:03.934638 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f7ad6d9a-4bb1-4c82-8777-a92e1f15d3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:15:03.946287 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad6d9a-4bb1-4c82-8777-a92e1f15d3ef None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:15:03.949961 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff3073f-5ffe-4d81-a4b4-6673b6f583ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:15:03.951227 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3073f-5ffe-4d81-a4b4-6673b6f583ac service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:03.951607 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3073f-5ffe-4d81-a4b4-6673b6f583ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:03.951948 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3073f-5ffe-4d81-a4b4-6673b6f583ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:03.953709 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9ff3073f-5ffe-4d81-a4b4-6673b6f583ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:15:04.013678 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff3073f-5ffe-4d81-a4b4-6673b6f583ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:15:04.015097 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff3073f-5ffe-4d81-a4b4-6673b6f583ac service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:15:04.023878 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1316/2636] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:15:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:15:14.019903 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=28 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:15:14.617772 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20861d-a3a9-46da-8b2d-a229e0a49f8f None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:14.618278 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20861d-a3a9-46da-8b2d-a229e0a49f8f None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:14.618487 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20861d-a3a9-46da-8b2d-a229e0a49f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:14.647029 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-af20861d-a3a9-46da-8b2d-a229e0a49f8f None None] MFA Rules not processed for user `d6d01ca673a1404c9b4cea51036c595c`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:15:14.648495 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-af20861d-a3a9-46da-8b2d-a229e0a49f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:15:14.663855 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1321/2637] 10.222.0.27 () {58 vars in 967 bytes} [Sun Sep 4 01:15:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:15:14.672136 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff3073f-5ffe-4d81-a4b4-6673b6f583ac service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:15:14.687367 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff3073f-5ffe-4d81-a4b4-6673b6f583ac service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:15:14.699914 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3277d3-8f3a-48ea-bd62-0da8d7a56b11 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2297d8b3fe8e47eb9fd39a6e3eea8f54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:15:14.703361 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3277d3-8f3a-48ea-bd62-0da8d7a56b11 service swift] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:14.703361 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3277d3-8f3a-48ea-bd62-0da8d7a56b11 service swift] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:14.703361 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3277d3-8f3a-48ea-bd62-0da8d7a56b11 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:14.720533 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3277d3-8f3a-48ea-bd62-0da8d7a56b11 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:15:14.721848 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3277d3-8f3a-48ea-bd62-0da8d7a56b11 service swift] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:15:14.737751 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1317/2638] 10.222.0.27 () {60 vars in 1384 bytes} [Sun Sep 4 01:15:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 04 01:15:16.244835 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20861d-a3a9-46da-8b2d-a229e0a49f8f None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:15:16.254862 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20861d-a3a9-46da-8b2d-a229e0a49f8f None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:15:16.258441 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8195953-62bd-437b-8728-d16aa0827f8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:15:16.259034 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8195953-62bd-437b-8728-d16aa0827f8d service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:16.259309 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8195953-62bd-437b-8728-d16aa0827f8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:16.259577 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8195953-62bd-437b-8728-d16aa0827f8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:16.261566 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b8195953-62bd-437b-8728-d16aa0827f8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:15:16.325037 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8195953-62bd-437b-8728-d16aa0827f8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:15:16.327154 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8195953-62bd-437b-8728-d16aa0827f8d service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:15:16.336870 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1322/2639] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:15:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:15:16.659772 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b17bb-5470-4555-834b-4a4cabf9c945 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:16.660293 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b17bb-5470-4555-834b-4a4cabf9c945 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:16.660591 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b17bb-5470-4555-834b-4a4cabf9c945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:16.681484 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-125b17bb-5470-4555-834b-4a4cabf9c945 None None] MFA Rules not processed for user `d6d01ca673a1404c9b4cea51036c595c`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:15:16.682719 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-125b17bb-5470-4555-834b-4a4cabf9c945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:15:16.691030 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1318/2640] 10.222.0.27 () {58 vars in 1009 bytes} [Sun Sep 4 01:15:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:15:16.704518 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8195953-62bd-437b-8728-d16aa0827f8d service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:15:16.715881 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8195953-62bd-437b-8728-d16aa0827f8d service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:15:16.721887 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a90575-2bef-498c-8b92-f317cb67419b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2297d8b3fe8e47eb9fd39a6e3eea8f54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:15:16.722767 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a90575-2bef-498c-8b92-f317cb67419b service swift] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:16.723042 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a90575-2bef-498c-8b92-f317cb67419b service swift] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:16.723493 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a90575-2bef-498c-8b92-f317cb67419b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:16.732218 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a90575-2bef-498c-8b92-f317cb67419b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:15:16.733904 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a90575-2bef-498c-8b92-f317cb67419b service swift] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:15:16.742548 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1323/2641] 10.222.0.27 () {60 vars in 1384 bytes} [Sun Sep 4 01:15:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 04 01:15:17.682959 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86acba1-fa28-484c-a4e3-0ab3fb651749 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:17.683438 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86acba1-fa28-484c-a4e3-0ab3fb651749 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:17.683665 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86acba1-fa28-484c-a4e3-0ab3fb651749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:17.705570 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c86acba1-fa28-484c-a4e3-0ab3fb651749 None None] MFA Rules not processed for user `d6d01ca673a1404c9b4cea51036c595c`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:15:17.708451 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c86acba1-fa28-484c-a4e3-0ab3fb651749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:15:17.714483 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1319/2642] 10.222.0.27 () {58 vars in 967 bytes} [Sun Sep 4 01:15:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:15:17.727801 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a90575-2bef-498c-8b92-f317cb67419b service swift] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:15:17.738822 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a90575-2bef-498c-8b92-f317cb67419b service swift] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:15:17.744047 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc7022e-c58e-427f-ad9f-8b69f4524fe0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2297d8b3fe8e47eb9fd39a6e3eea8f54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:15:17.744694 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc7022e-c58e-427f-ad9f-8b69f4524fe0 service swift] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:17.744977 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc7022e-c58e-427f-ad9f-8b69f4524fe0 service swift] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:17.745253 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc7022e-c58e-427f-ad9f-8b69f4524fe0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:17.754564 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc7022e-c58e-427f-ad9f-8b69f4524fe0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:15:17.756204 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc7022e-c58e-427f-ad9f-8b69f4524fe0 service swift] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:15:17.766465 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1324/2643] 10.222.0.27 () {60 vars in 1384 bytes} [Sun Sep 4 01:15:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 04 01:15:18.378025 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2897487-f773-479f-98ff-3f7cbc9b7ac5 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:18.378315 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2897487-f773-479f-98ff-3f7cbc9b7ac5 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:18.378601 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2897487-f773-479f-98ff-3f7cbc9b7ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:18.400804 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-a2897487-f773-479f-98ff-3f7cbc9b7ac5 None None] MFA Rules not processed for user `d6d01ca673a1404c9b4cea51036c595c`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:15:18.402204 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a2897487-f773-479f-98ff-3f7cbc9b7ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:15:18.412880 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1320/2644] 10.222.0.27 () {58 vars in 1009 bytes} [Sun Sep 4 01:15:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:15:18.427156 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc7022e-c58e-427f-ad9f-8b69f4524fe0 service swift] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:15:18.439526 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc7022e-c58e-427f-ad9f-8b69f4524fe0 service swift] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:15:18.447434 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4324ed-e5cd-4b64-b0e2-9c700fb4e70f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2297d8b3fe8e47eb9fd39a6e3eea8f54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:15:18.454684 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4324ed-e5cd-4b64-b0e2-9c700fb4e70f service swift] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:18.454684 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4324ed-e5cd-4b64-b0e2-9c700fb4e70f service swift] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:18.454684 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4324ed-e5cd-4b64-b0e2-9c700fb4e70f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:18.458667 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4324ed-e5cd-4b64-b0e2-9c700fb4e70f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:15:18.460591 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4324ed-e5cd-4b64-b0e2-9c700fb4e70f service swift] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:15:18.473781 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1325/2645] 10.222.0.27 () {60 vars in 1384 bytes} [Sun Sep 4 01:15:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 04 01:15:19.009589 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ffbf1-c660-4514-a414-6bf05a1ce4d1 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:19.010315 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ffbf1-c660-4514-a414-6bf05a1ce4d1 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:19.010653 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ffbf1-c660-4514-a414-6bf05a1ce4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:19.033674 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-e28ffbf1-c660-4514-a414-6bf05a1ce4d1 None None] MFA Rules not processed for user `d6d01ca673a1404c9b4cea51036c595c`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:15:19.034907 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e28ffbf1-c660-4514-a414-6bf05a1ce4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:15:19.042544 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1321/2646] 10.222.0.27 () {58 vars in 967 bytes} [Sun Sep 4 01:15:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:15:19.056421 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4324ed-e5cd-4b64-b0e2-9c700fb4e70f service swift] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:15:19.071068 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4324ed-e5cd-4b64-b0e2-9c700fb4e70f service swift] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:15:19.076232 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8660ffeb-5bd8-4248-aaf8-63d4983315b7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2297d8b3fe8e47eb9fd39a6e3eea8f54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:15:19.077025 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660ffeb-5bd8-4248-aaf8-63d4983315b7 service swift] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:19.077555 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660ffeb-5bd8-4248-aaf8-63d4983315b7 service swift] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:19.077842 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660ffeb-5bd8-4248-aaf8-63d4983315b7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:19.086869 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8660ffeb-5bd8-4248-aaf8-63d4983315b7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:15:19.088665 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8660ffeb-5bd8-4248-aaf8-63d4983315b7 service swift] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:15:19.098408 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1326/2647] 10.222.0.27 () {60 vars in 1384 bytes} [Sun Sep 4 01:15:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 04 01:15:21.820877 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ffbf1-c660-4514-a414-6bf05a1ce4d1 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:15:21.833667 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ffbf1-c660-4514-a414-6bf05a1ce4d1 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:15:21.840576 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caa132b-5fbd-40c9-b18d-39302486bb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:15:21.841227 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caa132b-5fbd-40c9-b18d-39302486bb6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:21.841505 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caa132b-5fbd-40c9-b18d-39302486bb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:21.842626 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caa132b-5fbd-40c9-b18d-39302486bb6d admin admin] PATH_INFO: `/v3/users/2c9fe988cac442bfa1549d9b183498c5` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:21.844354 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caa132b-5fbd-40c9-b18d-39302486bb6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c9fe988cac442bfa1549d9b183498c5)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:15:21.846477 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caa132b-5fbd-40c9-b18d-39302486bb6d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:15:22.058592 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7caa132b-5fbd-40c9-b18d-39302486bb6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c9fe988cac442bfa1549d9b183498c5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:22.059162 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7caa132b-5fbd-40c9-b18d-39302486bb6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c9fe988cac442bfa1549d9b183498c5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:22.068542 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7caa132b-5fbd-40c9-b18d-39302486bb6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c9fe988cac442bfa1549d9b183498c5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:22.138746 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7caa132b-5fbd-40c9-b18d-39302486bb6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c9fe988cac442bfa1549d9b183498c5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:22.140180 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7caa132b-5fbd-40c9-b18d-39302486bb6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c9fe988cac442bfa1549d9b183498c5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:22.163214 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1322/2648] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:15:21 2022] DELETE /identity/v3/users/2c9fe988cac442bfa1549d9b183498c5 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:15:22.167523 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedf5eb-2110-4d68-bcca-95bfa160d555 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:22.168013 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedf5eb-2110-4d68-bcca-95bfa160d555 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:22.168119 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedf5eb-2110-4d68-bcca-95bfa160d555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:22.189242 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-1eedf5eb-2110-4d68-bcca-95bfa160d555 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:15:22.190472 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-1eedf5eb-2110-4d68-bcca-95bfa160d555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:15:22.267271 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1327/2649] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:15:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:15:22.308176 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caa132b-5fbd-40c9-b18d-39302486bb6d admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:15:22.309826 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7caa132b-5fbd-40c9-b18d-39302486bb6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:15:22.369886 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caa132b-5fbd-40c9-b18d-39302486bb6d admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:15:22.373598 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a269679-3a1a-4689-b499-3b4195790e34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:15:22.374201 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a269679-3a1a-4689-b499-3b4195790e34 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:22.374479 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a269679-3a1a-4689-b499-3b4195790e34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:22.374749 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a269679-3a1a-4689-b499-3b4195790e34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:22.383859 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a269679-3a1a-4689-b499-3b4195790e34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:15:22.388775 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a269679-3a1a-4689-b499-3b4195790e34 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:15:22.395108 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1323/2650] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:15:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:15:22.838373 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eedf5eb-2110-4d68-bcca-95bfa160d555 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:15:22.840102 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-1eedf5eb-2110-4d68-bcca-95bfa160d555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:15:22.853665 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eedf5eb-2110-4d68-bcca-95bfa160d555 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:15:22.858991 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dbbc7a-35a9-461b-bafb-efb930fb46c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:15:22.859866 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbbc7a-35a9-461b-bafb-efb930fb46c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:22.860147 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbbc7a-35a9-461b-bafb-efb930fb46c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:22.860420 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbbc7a-35a9-461b-bafb-efb930fb46c2 admin admin] PATH_INFO: `/v3/projects/db1b33cc7cd2466b9f9cf4e0a631941e` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:22.861653 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dbbc7a-35a9-461b-bafb-efb930fb46c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db1b33cc7cd2466b9f9cf4e0a631941e)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:15:22.863688 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dbbc7a-35a9-461b-bafb-efb930fb46c2 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:15:22.989353 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:15:23.003017 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-05dbbc7a-35a9-461b-bafb-efb930fb46c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db1b33cc7cd2466b9f9cf4e0a631941e', 'request_id': 'req-05dbbc7a-35a9-461b-bafb-efb930fb46c2'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:23.070162 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-05dbbc7a-35a9-461b-bafb-efb930fb46c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db1b33cc7cd2466b9f9cf4e0a631941e', 'request_id': 'req-05dbbc7a-35a9-461b-bafb-efb930fb46c2'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:23.076602 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-05dbbc7a-35a9-461b-bafb-efb930fb46c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db1b33cc7cd2466b9f9cf4e0a631941e', 'request_id': 'req-05dbbc7a-35a9-461b-bafb-efb930fb46c2'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:23.087419 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-05dbbc7a-35a9-461b-bafb-efb930fb46c2 admin admin] The token cache is being invalidate because project db1b33cc7cd2466b9f9cf4e0a631941e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:15:23.087705 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-05dbbc7a-35a9-461b-bafb-efb930fb46c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:23.088329 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-05dbbc7a-35a9-461b-bafb-efb930fb46c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:23.089863 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1328/2651] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:15:22 2022] DELETE /identity/v3/projects/db1b33cc7cd2466b9f9cf4e0a631941e => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:15:23.922096 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a269679-3a1a-4689-b499-3b4195790e34 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:15:23.923611 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6a269679-3a1a-4689-b499-3b4195790e34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:15:23.937331 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a269679-3a1a-4689-b499-3b4195790e34 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:15:23.969804 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:15:23.993460 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a08139-4b9c-4895-8e7f-e8f24db93db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:15:23.994080 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a08139-4b9c-4895-8e7f-e8f24db93db2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:23.994372 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a08139-4b9c-4895-8e7f-e8f24db93db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:23.994848 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a08139-4b9c-4895-8e7f-e8f24db93db2 admin admin] PATH_INFO: `/v3/users/7fee1d5bc3da421fa1c85ea59223fb54` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:23.996515 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a08139-4b9c-4895-8e7f-e8f24db93db2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fee1d5bc3da421fa1c85ea59223fb54)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:15:23.998726 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a08139-4b9c-4895-8e7f-e8f24db93db2 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:15:24.169322 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-89a08139-4b9c-4895-8e7f-e8f24db93db2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fee1d5bc3da421fa1c85ea59223fb54'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:24.169961 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-89a08139-4b9c-4895-8e7f-e8f24db93db2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fee1d5bc3da421fa1c85ea59223fb54'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:24.176975 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-89a08139-4b9c-4895-8e7f-e8f24db93db2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fee1d5bc3da421fa1c85ea59223fb54'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:24.246911 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-89a08139-4b9c-4895-8e7f-e8f24db93db2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fee1d5bc3da421fa1c85ea59223fb54'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:24.247531 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-89a08139-4b9c-4895-8e7f-e8f24db93db2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fee1d5bc3da421fa1c85ea59223fb54'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:24.265926 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1324/2652] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:15:23 2022] DELETE /identity/v3/users/7fee1d5bc3da421fa1c85ea59223fb54 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:15:24.534345 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dbbc7a-35a9-461b-bafb-efb930fb46c2 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:15:24.535826 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-05dbbc7a-35a9-461b-bafb-efb930fb46c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:15:24.550230 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dbbc7a-35a9-461b-bafb-efb930fb46c2 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:15:24.609523 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e097d47d-7ce9-4246-b7c8-b6ea4f33dab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:15:24.610154 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e097d47d-7ce9-4246-b7c8-b6ea4f33dab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:24.610432 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e097d47d-7ce9-4246-b7c8-b6ea4f33dab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:24.610707 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e097d47d-7ce9-4246-b7c8-b6ea4f33dab9 admin admin] PATH_INFO: `/v3/projects/be21c8fcaef64d9dbecf31bd633559c4` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:24.611808 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e097d47d-7ce9-4246-b7c8-b6ea4f33dab9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be21c8fcaef64d9dbecf31bd633559c4)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:15:24.613757 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e097d47d-7ce9-4246-b7c8-b6ea4f33dab9 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:15:24.715561 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e097d47d-7ce9-4246-b7c8-b6ea4f33dab9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be21c8fcaef64d9dbecf31bd633559c4', 'request_id': 'req-e097d47d-7ce9-4246-b7c8-b6ea4f33dab9'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:24.778185 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e097d47d-7ce9-4246-b7c8-b6ea4f33dab9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be21c8fcaef64d9dbecf31bd633559c4', 'request_id': 'req-e097d47d-7ce9-4246-b7c8-b6ea4f33dab9'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:24.784052 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e097d47d-7ce9-4246-b7c8-b6ea4f33dab9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be21c8fcaef64d9dbecf31bd633559c4', 'request_id': 'req-e097d47d-7ce9-4246-b7c8-b6ea4f33dab9'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:24.795792 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e097d47d-7ce9-4246-b7c8-b6ea4f33dab9 admin admin] The token cache is being invalidate because project be21c8fcaef64d9dbecf31bd633559c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:15:24.796092 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e097d47d-7ce9-4246-b7c8-b6ea4f33dab9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:24.796791 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e097d47d-7ce9-4246-b7c8-b6ea4f33dab9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:24.798308 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1329/2653] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:15:24 2022] DELETE /identity/v3/projects/be21c8fcaef64d9dbecf31bd633559c4 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:15:24.802500 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a08139-4b9c-4895-8e7f-e8f24db93db2 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:15:24.803896 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-89a08139-4b9c-4895-8e7f-e8f24db93db2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:15:24.817399 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a08139-4b9c-4895-8e7f-e8f24db93db2 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:15:24.872372 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c696c2-4792-4746-9a7a-99c92aeebedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:15:24.873662 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c696c2-4792-4746-9a7a-99c92aeebedc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:24.873940 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c696c2-4792-4746-9a7a-99c92aeebedc admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:24.874213 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c696c2-4792-4746-9a7a-99c92aeebedc admin admin] PATH_INFO: `/v3/users/8bea0cb76e174890939d3558681259f4` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:24.875321 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c696c2-4792-4746-9a7a-99c92aeebedc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bea0cb76e174890939d3558681259f4)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:15:24.877276 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c696c2-4792-4746-9a7a-99c92aeebedc admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:15:25.026060 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-03c696c2-4792-4746-9a7a-99c92aeebedc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bea0cb76e174890939d3558681259f4'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:25.026727 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-03c696c2-4792-4746-9a7a-99c92aeebedc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bea0cb76e174890939d3558681259f4'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:25.034395 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-03c696c2-4792-4746-9a7a-99c92aeebedc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bea0cb76e174890939d3558681259f4'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:25.036029 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:15:25.132198 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-03c696c2-4792-4746-9a7a-99c92aeebedc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bea0cb76e174890939d3558681259f4'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:25.132841 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-03c696c2-4792-4746-9a7a-99c92aeebedc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bea0cb76e174890939d3558681259f4'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:25.151326 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1325/2654] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:15:24 2022] DELETE /identity/v3/users/8bea0cb76e174890939d3558681259f4 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:15:25.431147 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e097d47d-7ce9-4246-b7c8-b6ea4f33dab9 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:15:25.432702 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e097d47d-7ce9-4246-b7c8-b6ea4f33dab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:15:25.446701 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e097d47d-7ce9-4246-b7c8-b6ea4f33dab9 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:15:25.498478 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:15:25.510347 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c2b522-bf8b-4031-96ad-b1b3d13f4b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:15:25.514062 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2b522-bf8b-4031-96ad-b1b3d13f4b92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:25.514062 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2b522-bf8b-4031-96ad-b1b3d13f4b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:25.514062 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2b522-bf8b-4031-96ad-b1b3d13f4b92 admin admin] PATH_INFO: `/v3/projects/07adc8f73fe240fd87ad0c8523874965` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:25.514062 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c2b522-bf8b-4031-96ad-b1b3d13f4b92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07adc8f73fe240fd87ad0c8523874965)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:15:25.514722 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c2b522-bf8b-4031-96ad-b1b3d13f4b92 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:15:25.630485 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a7c2b522-bf8b-4031-96ad-b1b3d13f4b92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07adc8f73fe240fd87ad0c8523874965', 'request_id': 'req-a7c2b522-bf8b-4031-96ad-b1b3d13f4b92'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:25.704146 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a7c2b522-bf8b-4031-96ad-b1b3d13f4b92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07adc8f73fe240fd87ad0c8523874965', 'request_id': 'req-a7c2b522-bf8b-4031-96ad-b1b3d13f4b92'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:25.709785 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a7c2b522-bf8b-4031-96ad-b1b3d13f4b92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07adc8f73fe240fd87ad0c8523874965', 'request_id': 'req-a7c2b522-bf8b-4031-96ad-b1b3d13f4b92'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:25.720976 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a7c2b522-bf8b-4031-96ad-b1b3d13f4b92 admin admin] The token cache is being invalidate because project 07adc8f73fe240fd87ad0c8523874965 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:15:25.725964 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a7c2b522-bf8b-4031-96ad-b1b3d13f4b92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:25.725964 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-a7c2b522-bf8b-4031-96ad-b1b3d13f4b92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:25.725964 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1330/2655] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:15:25 2022] DELETE /identity/v3/projects/07adc8f73fe240fd87ad0c8523874965 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:15:35.395482 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=8 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:15:35.718088 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=29,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:15:45.880859 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c696c2-4792-4746-9a7a-99c92aeebedc admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:15:45.882278 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-03c696c2-4792-4746-9a7a-99c92aeebedc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:15:45.938685 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c696c2-4792-4746-9a7a-99c92aeebedc admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:15:45.986868 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754a7dc-a4d1-4587-943b-710fb1b36b30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:15:45.987997 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754a7dc-a4d1-4587-943b-710fb1b36b30 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:45.988283 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754a7dc-a4d1-4587-943b-710fb1b36b30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:45.988587 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754a7dc-a4d1-4587-943b-710fb1b36b30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:45.990328 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7754a7dc-a4d1-4587-943b-710fb1b36b30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:15:46.053208 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7754a7dc-a4d1-4587-943b-710fb1b36b30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:15:46.054542 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7754a7dc-a4d1-4587-943b-710fb1b36b30 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:15:46.064066 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1326/2656] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:15:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:15:49.745038 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c2b522-bf8b-4031-96ad-b1b3d13f4b92 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:15:49.746520 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a7c2b522-bf8b-4031-96ad-b1b3d13f4b92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:15:49.760535 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c2b522-bf8b-4031-96ad-b1b3d13f4b92 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:15:49.765255 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83658076-91c5-4e31-b4c8-a7bd4005e869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:15:49.765871 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83658076-91c5-4e31-b4c8-a7bd4005e869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:49.766151 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83658076-91c5-4e31-b4c8-a7bd4005e869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:49.766425 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83658076-91c5-4e31-b4c8-a7bd4005e869 admin admin] PATH_INFO: `/v3/users/0c5e9731f6204b44841b8524762d5a77` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:49.767568 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83658076-91c5-4e31-b4c8-a7bd4005e869 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c5e9731f6204b44841b8524762d5a77)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:15:49.770335 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83658076-91c5-4e31-b4c8-a7bd4005e869 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:15:49.922684 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-83658076-91c5-4e31-b4c8-a7bd4005e869 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c5e9731f6204b44841b8524762d5a77'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:49.923326 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-83658076-91c5-4e31-b4c8-a7bd4005e869 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c5e9731f6204b44841b8524762d5a77'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:50.006956 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-83658076-91c5-4e31-b4c8-a7bd4005e869 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c5e9731f6204b44841b8524762d5a77'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:50.007575 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-83658076-91c5-4e31-b4c8-a7bd4005e869 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c5e9731f6204b44841b8524762d5a77'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:50.021008 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-83658076-91c5-4e31-b4c8-a7bd4005e869 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c5e9731f6204b44841b8524762d5a77'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:50.032934 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1331/2657] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:15:49 2022] DELETE /identity/v3/users/0c5e9731f6204b44841b8524762d5a77 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:15:50.265917 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754a7dc-a4d1-4587-943b-710fb1b36b30 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:15:50.279340 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754a7dc-a4d1-4587-943b-710fb1b36b30 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:15:50.335260 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ab4bf-6393-40a9-9f52-fdcfe3b46fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:15:50.335893 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ab4bf-6393-40a9-9f52-fdcfe3b46fef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:50.336173 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ab4bf-6393-40a9-9f52-fdcfe3b46fef admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:50.336450 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ab4bf-6393-40a9-9f52-fdcfe3b46fef admin admin] PATH_INFO: `/v3/projects/ef5de4321f9c4687a0bae83183b59a39` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:50.337580 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8ab4bf-6393-40a9-9f52-fdcfe3b46fef admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef5de4321f9c4687a0bae83183b59a39)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:15:50.339469 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8ab4bf-6393-40a9-9f52-fdcfe3b46fef admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:15:50.347856 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:15:50.464758 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6d8ab4bf-6393-40a9-9f52-fdcfe3b46fef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef5de4321f9c4687a0bae83183b59a39', 'request_id': 'req-6d8ab4bf-6393-40a9-9f52-fdcfe3b46fef'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:50.549924 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6d8ab4bf-6393-40a9-9f52-fdcfe3b46fef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef5de4321f9c4687a0bae83183b59a39', 'request_id': 'req-6d8ab4bf-6393-40a9-9f52-fdcfe3b46fef'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:50.555628 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6d8ab4bf-6393-40a9-9f52-fdcfe3b46fef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef5de4321f9c4687a0bae83183b59a39', 'request_id': 'req-6d8ab4bf-6393-40a9-9f52-fdcfe3b46fef'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:50.567225 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6d8ab4bf-6393-40a9-9f52-fdcfe3b46fef admin admin] The token cache is being invalidate because project ef5de4321f9c4687a0bae83183b59a39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:15:50.571153 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6d8ab4bf-6393-40a9-9f52-fdcfe3b46fef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:50.571153 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6d8ab4bf-6393-40a9-9f52-fdcfe3b46fef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:15:50.571153 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1327/2658] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:15:50 2022] DELETE /identity/v3/projects/ef5de4321f9c4687a0bae83183b59a39 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:15:50.574512 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ffcc4-22eb-43a3-b829-23af17e27e0a None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:50.574796 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ffcc4-22eb-43a3-b829-23af17e27e0a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:50.575070 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ffcc4-22eb-43a3-b829-23af17e27e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:50.595031 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-eb6ffcc4-22eb-43a3-b829-23af17e27e0a None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:15:50.596280 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-eb6ffcc4-22eb-43a3-b829-23af17e27e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:15:50.655016 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1332/2659] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:15:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:15:50.663296 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbdfa7-cd94-4888-9142-e48e16e00295 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:50.663584 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbdfa7-cd94-4888-9142-e48e16e00295 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:50.663970 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbdfa7-cd94-4888-9142-e48e16e00295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:50.683520 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-d1fbdfa7-cd94-4888-9142-e48e16e00295 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:15:50.684769 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d1fbdfa7-cd94-4888-9142-e48e16e00295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:15:50.693301 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1328/2660] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:15:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:15:50.697477 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ffcc4-22eb-43a3-b829-23af17e27e0a None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:15:50.707921 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ffcc4-22eb-43a3-b829-23af17e27e0a None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:15:50.712427 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2123396f-c761-4644-bc43-bf65eb88ec2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:15:50.713051 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123396f-c761-4644-bc43-bf65eb88ec2c admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:50.713334 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123396f-c761-4644-bc43-bf65eb88ec2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:50.713609 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123396f-c761-4644-bc43-bf65eb88ec2c admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:50.714255 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2123396f-c761-4644-bc43-bf65eb88ec2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:15:50.714558 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2123396f-c761-4644-bc43-bf65eb88ec2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:15:50.716153 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2123396f-c761-4644-bc43-bf65eb88ec2c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:15:50.727772 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1333/2661] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:15:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:15:50.731940 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fbdfa7-cd94-4888-9142-e48e16e00295 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:15:50.741203 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fbdfa7-cd94-4888-9142-e48e16e00295 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:15:50.745874 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1bdeb0-5947-43bb-a9f6-bb92de206639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:15:50.746460 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1bdeb0-5947-43bb-a9f6-bb92de206639 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:50.746742 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1bdeb0-5947-43bb-a9f6-bb92de206639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:50.747015 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1bdeb0-5947-43bb-a9f6-bb92de206639 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:50.747669 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1bdeb0-5947-43bb-a9f6-bb92de206639 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:15:50.749600 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1bdeb0-5947-43bb-a9f6-bb92de206639 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:15:50.837715 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1329/2662] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:15:50 2022] POST /identity/v3/projects => generated 371 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:15:50.841980 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2123396f-c761-4644-bc43-bf65eb88ec2c admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:15:50.852209 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2123396f-c761-4644-bc43-bf65eb88ec2c admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:15:50.908215 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e1ef7-95b1-4d82-aaae-ebbd59f9e51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:15:50.908842 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e1ef7-95b1-4d82-aaae-ebbd59f9e51c admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:50.909153 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e1ef7-95b1-4d82-aaae-ebbd59f9e51c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:50.909638 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e1ef7-95b1-4d82-aaae-ebbd59f9e51c admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:50.910321 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e1ef7-95b1-4d82-aaae-ebbd59f9e51c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:15:50.912233 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e1ef7-95b1-4d82-aaae-ebbd59f9e51c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:15:50.920087 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:15:51.001229 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1334/2663] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:15:50 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:15:51.006394 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1bdeb0-5947-43bb-a9f6-bb92de206639 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:15:51.016990 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1bdeb0-5947-43bb-a9f6-bb92de206639 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:15:51.021959 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f5a34f-753e-4dd9-9940-4f026362b527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:15:51.022550 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5a34f-753e-4dd9-9940-4f026362b527 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:51.022827 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5a34f-753e-4dd9-9940-4f026362b527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:51.023097 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5a34f-753e-4dd9-9940-4f026362b527 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:51.023724 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f5a34f-753e-4dd9-9940-4f026362b527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:15:51.025360 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f5a34f-753e-4dd9-9940-4f026362b527 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:15:51.035261 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1330/2664] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:15:51.039293 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e1ef7-95b1-4d82-aaae-ebbd59f9e51c admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:15:51.050273 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e1ef7-95b1-4d82-aaae-ebbd59f9e51c admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:15:51.055369 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bbfad2-328c-43d9-bd15-720d5c9f4939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:15:51.055961 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbfad2-328c-43d9-bd15-720d5c9f4939 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:51.056244 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbfad2-328c-43d9-bd15-720d5c9f4939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:51.056551 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbfad2-328c-43d9-bd15-720d5c9f4939 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:51.057193 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bbfad2-328c-43d9-bd15-720d5c9f4939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:15:51.058782 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bbfad2-328c-43d9-bd15-720d5c9f4939 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:15:51.070198 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1335/2665] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:15:51.074451 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f5a34f-753e-4dd9-9940-4f026362b527 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:15:51.084205 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f5a34f-753e-4dd9-9940-4f026362b527 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:15:51.089524 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b66fa-4fb2-4bd9-9ae2-d7cecdd2ef44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:15:51.090121 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b66fa-4fb2-4bd9-9ae2-d7cecdd2ef44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:51.090404 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b66fa-4fb2-4bd9-9ae2-d7cecdd2ef44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:51.090677 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b66fa-4fb2-4bd9-9ae2-d7cecdd2ef44 admin admin] PATH_INFO: `/v3/projects/8cfa45fad83d478eac535a6d435ed165/users/a990f3140b314d95b567fae1f6da2ba4/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:51.105204 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2b66fa-4fb2-4bd9-9ae2-d7cecdd2ef44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cfa45fad83d478eac535a6d435ed165, user_id=a990f3140b314d95b567fae1f6da2ba4, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:15:51.107842 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2b66fa-4fb2-4bd9-9ae2-d7cecdd2ef44 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:15:51.166778 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1331/2666] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:15:51 2022] PUT /identity/v3/projects/8cfa45fad83d478eac535a6d435ed165/users/a990f3140b314d95b567fae1f6da2ba4/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:15:51.177557 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c15c5e-e808-4d15-b70b-40a6c2935f06 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:51.177845 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c15c5e-e808-4d15-b70b-40a6c2935f06 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:51.178126 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c15c5e-e808-4d15-b70b-40a6c2935f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:51.205805 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-30c15c5e-e808-4d15-b70b-40a6c2935f06 None None] MFA Rules not processed for user `a990f3140b314d95b567fae1f6da2ba4`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:15:51.207056 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-30c15c5e-e808-4d15-b70b-40a6c2935f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:15:51.272280 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1336/2667] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:15:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:15:51.277025 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b66fa-4fb2-4bd9-9ae2-d7cecdd2ef44 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:15:51.340354 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b66fa-4fb2-4bd9-9ae2-d7cecdd2ef44 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:15:51.345000 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98467aa-34de-4285-9add-4f4503007bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:15:51.345602 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98467aa-34de-4285-9add-4f4503007bd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:51.345887 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98467aa-34de-4285-9add-4f4503007bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:51.346163 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98467aa-34de-4285-9add-4f4503007bd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:51.347340 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98467aa-34de-4285-9add-4f4503007bd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:15:51.349260 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98467aa-34de-4285-9add-4f4503007bd2 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:15:51.408197 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1332/2668] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:15:51 2022] POST /identity/v3/projects => generated 371 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:15:51.412944 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c15c5e-e808-4d15-b70b-40a6c2935f06 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:15:51.423942 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c15c5e-e808-4d15-b70b-40a6c2935f06 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:15:51.481093 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c469faa-253d-4553-91a2-9429a53203ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:15:51.481695 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c469faa-253d-4553-91a2-9429a53203ed admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:51.481975 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c469faa-253d-4553-91a2-9429a53203ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:51.482247 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c469faa-253d-4553-91a2-9429a53203ed admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:51.482867 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c469faa-253d-4553-91a2-9429a53203ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:15:51.484816 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c469faa-253d-4553-91a2-9429a53203ed admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:15:51.538638 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1337/2669] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:15:51 2022] POST /identity/v3/users => generated 292 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:15:51.542666 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98467aa-34de-4285-9add-4f4503007bd2 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:15:51.553798 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98467aa-34de-4285-9add-4f4503007bd2 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:15:51.558443 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f749616-2233-40cb-a5dc-100f5ef337b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:15:51.559032 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f749616-2233-40cb-a5dc-100f5ef337b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:51.559309 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f749616-2233-40cb-a5dc-100f5ef337b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:51.559577 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f749616-2233-40cb-a5dc-100f5ef337b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:51.560202 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f749616-2233-40cb-a5dc-100f5ef337b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:15:51.561841 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f749616-2233-40cb-a5dc-100f5ef337b2 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:15:51.572588 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1333/2670] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:15:51.576837 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c469faa-253d-4553-91a2-9429a53203ed admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:15:51.586861 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c469faa-253d-4553-91a2-9429a53203ed admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:15:51.591774 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362cbc1-f1a4-4cb2-b87b-0c37ddbe37f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:15:51.592366 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362cbc1-f1a4-4cb2-b87b-0c37ddbe37f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:51.592760 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362cbc1-f1a4-4cb2-b87b-0c37ddbe37f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:51.593043 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362cbc1-f1a4-4cb2-b87b-0c37ddbe37f1 admin admin] PATH_INFO: `/v3/projects/8d5910cdc44f4de69c9b238bf8e80698/users/2682e742621643ae9b210c809ff5fafc/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:51.609276 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9362cbc1-f1a4-4cb2-b87b-0c37ddbe37f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d5910cdc44f4de69c9b238bf8e80698, user_id=2682e742621643ae9b210c809ff5fafc, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:15:51.611916 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9362cbc1-f1a4-4cb2-b87b-0c37ddbe37f1 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:15:51.669584 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1338/2671] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:15:51 2022] PUT /identity/v3/projects/8d5910cdc44f4de69c9b238bf8e80698/users/2682e742621643ae9b210c809ff5fafc/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:15:51.673513 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f749616-2233-40cb-a5dc-100f5ef337b2 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:15:51.683661 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f749616-2233-40cb-a5dc-100f5ef337b2 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:15:51.687317 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:15:51.740308 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b49b5-6acc-46d2-862e-14db62d8fae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:15:51.740936 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b49b5-6acc-46d2-862e-14db62d8fae3 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:51.741219 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b49b5-6acc-46d2-862e-14db62d8fae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:51.741494 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b49b5-6acc-46d2-862e-14db62d8fae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:51.742125 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7b49b5-6acc-46d2-862e-14db62d8fae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:15:51.743705 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7b49b5-6acc-46d2-862e-14db62d8fae3 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:15:51.753536 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1334/2672] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:15:51.757488 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362cbc1-f1a4-4cb2-b87b-0c37ddbe37f1 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:15:51.768015 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362cbc1-f1a4-4cb2-b87b-0c37ddbe37f1 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:15:51.773486 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49060c4b-b505-4794-a000-a4685df39b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:15:51.773938 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49060c4b-b505-4794-a000-a4685df39b2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:51.774221 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49060c4b-b505-4794-a000-a4685df39b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:51.774497 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49060c4b-b505-4794-a000-a4685df39b2d admin admin] PATH_INFO: `/v3/projects/8d5910cdc44f4de69c9b238bf8e80698/users/2682e742621643ae9b210c809ff5fafc/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:51.776410 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49060c4b-b505-4794-a000-a4685df39b2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d5910cdc44f4de69c9b238bf8e80698, user_id=2682e742621643ae9b210c809ff5fafc, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:15:51.779145 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49060c4b-b505-4794-a000-a4685df39b2d admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:15:51.790708 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1339/2673] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:15:51 2022] PUT /identity/v3/projects/8d5910cdc44f4de69c9b238bf8e80698/users/2682e742621643ae9b210c809ff5fafc/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:15:51.802245 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40ddda-4e9e-467a-a93d-46fea741fb64 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:51.802539 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40ddda-4e9e-467a-a93d-46fea741fb64 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:51.802822 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40ddda-4e9e-467a-a93d-46fea741fb64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:51.830320 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1c40ddda-4e9e-467a-a93d-46fea741fb64 None None] MFA Rules not processed for user `2682e742621643ae9b210c809ff5fafc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:15:51.832068 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1c40ddda-4e9e-467a-a93d-46fea741fb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:15:51.901685 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1335/2674] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:15:51 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:15:51.913782 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49060c4b-b505-4794-a000-a4685df39b2d admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:15:51.915759 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-49060c4b-b505-4794-a000-a4685df39b2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:15:51.939567 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:15:51.995871 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49060c4b-b505-4794-a000-a4685df39b2d admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:15:51.999540 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c2bec-d13b-43d8-abd5-a18f35568c06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:15:52.000137 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c2bec-d13b-43d8-abd5-a18f35568c06 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:52.000419 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c2bec-d13b-43d8-abd5-a18f35568c06 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:52.000722 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c2bec-d13b-43d8-abd5-a18f35568c06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:52.010970 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5c2bec-d13b-43d8-abd5-a18f35568c06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:15:52.012399 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5c2bec-d13b-43d8-abd5-a18f35568c06 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:15:52.022729 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1340/2675] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:15:51 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:15:52.192321 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c40ddda-4e9e-467a-a93d-46fea741fb64 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:15:52.202116 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c40ddda-4e9e-467a-a93d-46fea741fb64 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:15:52.205713 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c554b97-2ee0-4477-b830-96d8a0cf5226 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:15:52.207382 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c554b97-2ee0-4477-b830-96d8a0cf5226 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:15:52.207664 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c554b97-2ee0-4477-b830-96d8a0cf5226 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:15:52.207942 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c554b97-2ee0-4477-b830-96d8a0cf5226 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:15:52.265035 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c554b97-2ee0-4477-b830-96d8a0cf5226 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:15:52.266457 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c554b97-2ee0-4477-b830-96d8a0cf5226 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:15:52.276116 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1336/2676] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:15:52 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:16:02.017566 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=26 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:16:02.272099 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=88 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:16:20.991126 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c2bec-d13b-43d8-abd5-a18f35568c06 service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:20.993022 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-1c5c2bec-d13b-43d8-abd5-a18f35568c06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:16:21.050604 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c2bec-d13b-43d8-abd5-a18f35568c06 service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:21.054225 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be2974-2df1-4502-b940-019586eba543 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:21.054864 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be2974-2df1-4502-b940-019586eba543 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:21.055135 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be2974-2df1-4502-b940-019586eba543 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:21.055410 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be2974-2df1-4502-b940-019586eba543 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:21.057140 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-64be2974-2df1-4502-b940-019586eba543 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:16:21.122230 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be2974-2df1-4502-b940-019586eba543 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:21.122230 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be2974-2df1-4502-b940-019586eba543 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:21.136824 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1341/2677] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:16:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:16:24.483269 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c554b97-2ee0-4477-b830-96d8a0cf5226 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:24.484168 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2c554b97-2ee0-4477-b830-96d8a0cf5226 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:16:24.500211 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c554b97-2ee0-4477-b830-96d8a0cf5226 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:24.505045 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886030b4-8952-43bc-bfde-c896c7814ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:24.505886 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886030b4-8952-43bc-bfde-c896c7814ec5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:24.506162 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886030b4-8952-43bc-bfde-c896c7814ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:24.506430 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886030b4-8952-43bc-bfde-c896c7814ec5 admin admin] PATH_INFO: `/v3/users/bd1c41fcdc724a128117a5288b8559f5` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:24.507590 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886030b4-8952-43bc-bfde-c896c7814ec5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd1c41fcdc724a128117a5288b8559f5)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:24.510082 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886030b4-8952-43bc-bfde-c896c7814ec5 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:24.678238 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-886030b4-8952-43bc-bfde-c896c7814ec5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd1c41fcdc724a128117a5288b8559f5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:24.678836 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-886030b4-8952-43bc-bfde-c896c7814ec5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd1c41fcdc724a128117a5288b8559f5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:24.685685 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-886030b4-8952-43bc-bfde-c896c7814ec5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd1c41fcdc724a128117a5288b8559f5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:24.743837 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-886030b4-8952-43bc-bfde-c896c7814ec5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd1c41fcdc724a128117a5288b8559f5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:24.744481 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-886030b4-8952-43bc-bfde-c896c7814ec5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd1c41fcdc724a128117a5288b8559f5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:24.762158 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1337/2678] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:16:24 2022] DELETE /identity/v3/users/bd1c41fcdc724a128117a5288b8559f5 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:16:24.997674 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be2974-2df1-4502-b940-019586eba543 service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:25.007953 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be2974-2df1-4502-b940-019586eba543 service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:25.075987 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74062882-3cb5-4a3c-991d-9594bc2f45a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:25.076656 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74062882-3cb5-4a3c-991d-9594bc2f45a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:25.076941 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74062882-3cb5-4a3c-991d-9594bc2f45a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:25.077219 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74062882-3cb5-4a3c-991d-9594bc2f45a6 admin admin] PATH_INFO: `/v3/projects/e334b7aa2f1848c1999d86ac666fe65e` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:25.087865 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74062882-3cb5-4a3c-991d-9594bc2f45a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e334b7aa2f1848c1999d86ac666fe65e)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:25.089843 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74062882-3cb5-4a3c-991d-9594bc2f45a6 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:25.186520 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:16:25.190185 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-74062882-3cb5-4a3c-991d-9594bc2f45a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e334b7aa2f1848c1999d86ac666fe65e', 'request_id': 'req-74062882-3cb5-4a3c-991d-9594bc2f45a6'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:25.286551 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-74062882-3cb5-4a3c-991d-9594bc2f45a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e334b7aa2f1848c1999d86ac666fe65e', 'request_id': 'req-74062882-3cb5-4a3c-991d-9594bc2f45a6'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:25.293130 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-74062882-3cb5-4a3c-991d-9594bc2f45a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e334b7aa2f1848c1999d86ac666fe65e', 'request_id': 'req-74062882-3cb5-4a3c-991d-9594bc2f45a6'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:25.305887 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-74062882-3cb5-4a3c-991d-9594bc2f45a6 admin admin] The token cache is being invalidate because project e334b7aa2f1848c1999d86ac666fe65e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:16:25.306175 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-74062882-3cb5-4a3c-991d-9594bc2f45a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:25.306816 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-74062882-3cb5-4a3c-991d-9594bc2f45a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:25.308295 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1342/2679] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:16:24 2022] DELETE /identity/v3/projects/e334b7aa2f1848c1999d86ac666fe65e => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:16:25.313566 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4af41-19a3-462d-bccc-08956f1f1352 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:25.313856 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4af41-19a3-462d-bccc-08956f1f1352 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:25.314134 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4af41-19a3-462d-bccc-08956f1f1352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:25.334104 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-77f4af41-19a3-462d-bccc-08956f1f1352 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:16:25.335459 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-77f4af41-19a3-462d-bccc-08956f1f1352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:16:25.398288 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1338/2680] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:16:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:16:25.407276 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee98f71a-07da-45c1-ac8f-5d252b797ea6 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:25.407564 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee98f71a-07da-45c1-ac8f-5d252b797ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:25.407873 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee98f71a-07da-45c1-ac8f-5d252b797ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:25.427381 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-ee98f71a-07da-45c1-ac8f-5d252b797ea6 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:16:25.428664 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ee98f71a-07da-45c1-ac8f-5d252b797ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:16:25.437290 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1343/2681] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:16:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:16:25.441648 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4af41-19a3-462d-bccc-08956f1f1352 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:25.452585 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4af41-19a3-462d-bccc-08956f1f1352 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:25.457128 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf32a15-00e1-4722-b803-b26c4e8f4d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:25.457761 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf32a15-00e1-4722-b803-b26c4e8f4d83 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:25.458048 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf32a15-00e1-4722-b803-b26c4e8f4d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:25.458324 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf32a15-00e1-4722-b803-b26c4e8f4d83 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:25.458973 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf32a15-00e1-4722-b803-b26c4e8f4d83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:16:25.459277 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf32a15-00e1-4722-b803-b26c4e8f4d83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:25.460935 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf32a15-00e1-4722-b803-b26c4e8f4d83 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:25.472718 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1339/2682] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:16:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:16:25.477188 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee98f71a-07da-45c1-ac8f-5d252b797ea6 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:25.487402 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee98f71a-07da-45c1-ac8f-5d252b797ea6 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:25.492794 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e828b33c-f577-476b-b589-cf2a393d2b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:25.493891 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828b33c-f577-476b-b589-cf2a393d2b29 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:25.494175 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828b33c-f577-476b-b589-cf2a393d2b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:25.494450 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828b33c-f577-476b-b589-cf2a393d2b29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:25.495100 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e828b33c-f577-476b-b589-cf2a393d2b29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:25.497113 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e828b33c-f577-476b-b589-cf2a393d2b29 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:25.614473 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1344/2683] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:16:25 2022] POST /identity/v3/projects => generated 377 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:16:25.619243 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf32a15-00e1-4722-b803-b26c4e8f4d83 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:25.629395 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf32a15-00e1-4722-b803-b26c4e8f4d83 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:25.685883 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d94524-17b3-4e99-8bdb-3a3e9b58abe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:25.686480 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d94524-17b3-4e99-8bdb-3a3e9b58abe7 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:25.686761 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d94524-17b3-4e99-8bdb-3a3e9b58abe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:25.687035 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d94524-17b3-4e99-8bdb-3a3e9b58abe7 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:25.687652 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d94524-17b3-4e99-8bdb-3a3e9b58abe7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:25.689587 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d94524-17b3-4e99-8bdb-3a3e9b58abe7 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:25.697475 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:16:25.800790 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1340/2684] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:16:25 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:16:25.805544 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e828b33c-f577-476b-b589-cf2a393d2b29 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:25.816384 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e828b33c-f577-476b-b589-cf2a393d2b29 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:25.821320 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5541c2-330c-40da-9a97-bc316f90d7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:25.821919 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5541c2-330c-40da-9a97-bc316f90d7ad admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:25.822227 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5541c2-330c-40da-9a97-bc316f90d7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:25.822528 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5541c2-330c-40da-9a97-bc316f90d7ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:25.823173 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5541c2-330c-40da-9a97-bc316f90d7ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:25.824801 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5541c2-330c-40da-9a97-bc316f90d7ad admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:25.834696 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1345/2685] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:16:25.839185 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d94524-17b3-4e99-8bdb-3a3e9b58abe7 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:25.849091 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d94524-17b3-4e99-8bdb-3a3e9b58abe7 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:25.854881 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f0741-8f2d-4688-8513-c0be70aae73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:25.855472 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f0741-8f2d-4688-8513-c0be70aae73a admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:25.855753 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f0741-8f2d-4688-8513-c0be70aae73a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:25.856032 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f0741-8f2d-4688-8513-c0be70aae73a admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:25.856703 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3f0741-8f2d-4688-8513-c0be70aae73a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:25.858281 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3f0741-8f2d-4688-8513-c0be70aae73a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:25.869740 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1341/2686] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:16:25.873896 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5541c2-330c-40da-9a97-bc316f90d7ad admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:25.883891 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5541c2-330c-40da-9a97-bc316f90d7ad admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:25.888747 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d60f38-507f-4362-8462-575484d81a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:25.889387 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d60f38-507f-4362-8462-575484d81a5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:25.889670 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d60f38-507f-4362-8462-575484d81a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:25.889950 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d60f38-507f-4362-8462-575484d81a5e admin admin] PATH_INFO: `/v3/projects/5cfeb662f87744b195af64361774c98a/users/c5819e21fbba41d3b235daf8f0d6769b/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:25.904784 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d60f38-507f-4362-8462-575484d81a5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cfeb662f87744b195af64361774c98a, user_id=c5819e21fbba41d3b235daf8f0d6769b, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:25.907431 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d60f38-507f-4362-8462-575484d81a5e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:25.978809 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1346/2687] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:16:25 2022] PUT /identity/v3/projects/5cfeb662f87744b195af64361774c98a/users/c5819e21fbba41d3b235daf8f0d6769b/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:16:25.984054 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6fbb40-c666-49c5-9e80-6b479f749cd6 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:25.984348 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6fbb40-c666-49c5-9e80-6b479f749cd6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:25.984652 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6fbb40-c666-49c5-9e80-6b479f749cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:26.004370 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-aa6fbb40-c666-49c5-9e80-6b479f749cd6 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:16:26.006240 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-aa6fbb40-c666-49c5-9e80-6b479f749cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:16:26.065476 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1342/2688] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:16:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:16:26.075199 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d60f38-507f-4362-8462-575484d81a5e admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:26.076920 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e4d60f38-507f-4362-8462-575484d81a5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:16:26.136126 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d60f38-507f-4362-8462-575484d81a5e admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:26.139944 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe417d2f-966a-41d2-a22a-8e05530d3eb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:26.140766 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe417d2f-966a-41d2-a22a-8e05530d3eb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:26.141050 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe417d2f-966a-41d2-a22a-8e05530d3eb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:26.141375 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe417d2f-966a-41d2-a22a-8e05530d3eb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:26.150493 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe417d2f-966a-41d2-a22a-8e05530d3eb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:26.151930 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe417d2f-966a-41d2-a22a-8e05530d3eb3 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:26.162725 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1347/2689] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:16:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:16:30.205102 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6fbb40-c666-49c5-9e80-6b479f749cd6 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:30.207079 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-aa6fbb40-c666-49c5-9e80-6b479f749cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:16:30.219518 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6fbb40-c666-49c5-9e80-6b479f749cd6 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:30.224403 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f8964-1b0e-4c54-827c-60820647926a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:30.225301 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f8964-1b0e-4c54-827c-60820647926a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:30.226371 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f8964-1b0e-4c54-827c-60820647926a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:30.233549 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f8964-1b0e-4c54-827c-60820647926a admin admin] PATH_INFO: `/v3/users/a990f3140b314d95b567fae1f6da2ba4` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:30.233549 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f8964-1b0e-4c54-827c-60820647926a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a990f3140b314d95b567fae1f6da2ba4)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:30.233549 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f8964-1b0e-4c54-827c-60820647926a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:30.422325 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c12f8964-1b0e-4c54-827c-60820647926a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a990f3140b314d95b567fae1f6da2ba4'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:30.422965 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c12f8964-1b0e-4c54-827c-60820647926a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a990f3140b314d95b567fae1f6da2ba4'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:30.430776 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c12f8964-1b0e-4c54-827c-60820647926a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a990f3140b314d95b567fae1f6da2ba4'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:30.529308 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c12f8964-1b0e-4c54-827c-60820647926a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a990f3140b314d95b567fae1f6da2ba4'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:30.535077 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c12f8964-1b0e-4c54-827c-60820647926a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a990f3140b314d95b567fae1f6da2ba4'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:30.556001 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1343/2690] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:16:30 2022] DELETE /identity/v3/users/a990f3140b314d95b567fae1f6da2ba4 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:16:30.560781 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7685ad-58cd-49ae-8251-1a0fe27abfbf None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:30.561256 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7685ad-58cd-49ae-8251-1a0fe27abfbf None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:30.561374 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7685ad-58cd-49ae-8251-1a0fe27abfbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:30.588085 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-3f7685ad-58cd-49ae-8251-1a0fe27abfbf None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:16:30.588085 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3f7685ad-58cd-49ae-8251-1a0fe27abfbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:16:30.810210 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1348/2691] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:16:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:16:30.822106 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f8964-1b0e-4c54-827c-60820647926a admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:30.823515 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c12f8964-1b0e-4c54-827c-60820647926a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:16:30.892763 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f8964-1b0e-4c54-827c-60820647926a admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:30.896472 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4ca34-765b-42dd-b26d-0f589c3dbd96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:30.897208 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4ca34-765b-42dd-b26d-0f589c3dbd96 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:30.897636 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4ca34-765b-42dd-b26d-0f589c3dbd96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:30.898186 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4ca34-765b-42dd-b26d-0f589c3dbd96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:30.904361 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:16:30.908255 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd4ca34-765b-42dd-b26d-0f589c3dbd96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:30.909878 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd4ca34-765b-42dd-b26d-0f589c3dbd96 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:30.921603 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1344/2692] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:16:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:16:31.464813 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7685ad-58cd-49ae-8251-1a0fe27abfbf None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:31.466280 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3f7685ad-58cd-49ae-8251-1a0fe27abfbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:16:31.474187 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:16:31.482867 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7685ad-58cd-49ae-8251-1a0fe27abfbf None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:31.501946 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084626b-9b36-4172-9220-43e25d823028 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:31.507562 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084626b-9b36-4172-9220-43e25d823028 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:31.507562 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084626b-9b36-4172-9220-43e25d823028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:31.507937 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f522b16-1208-4698-b988-4868830644c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:31.507937 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f522b16-1208-4698-b988-4868830644c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:31.507937 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f522b16-1208-4698-b988-4868830644c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:31.507937 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f522b16-1208-4698-b988-4868830644c5 admin admin] PATH_INFO: `/v3/projects/8cfa45fad83d478eac535a6d435ed165` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:31.508523 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f522b16-1208-4698-b988-4868830644c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cfa45fad83d478eac535a6d435ed165)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:31.508523 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f522b16-1208-4698-b988-4868830644c5 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:31.537500 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f084626b-9b36-4172-9220-43e25d823028 None None] MFA Rules not processed for user `c5819e21fbba41d3b235daf8f0d6769b`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:16:31.539984 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f084626b-9b36-4172-9220-43e25d823028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:16:31.656388 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-7f522b16-1208-4698-b988-4868830644c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cfa45fad83d478eac535a6d435ed165', 'request_id': 'req-7f522b16-1208-4698-b988-4868830644c5'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:31.667558 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1345/2693] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:16:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:16:31.677935 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f084626b-9b36-4172-9220-43e25d823028 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:31.678242 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f084626b-9b36-4172-9220-43e25d823028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:16:31.715969 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-7f522b16-1208-4698-b988-4868830644c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cfa45fad83d478eac535a6d435ed165', 'request_id': 'req-7f522b16-1208-4698-b988-4868830644c5'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:31.726874 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-7f522b16-1208-4698-b988-4868830644c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cfa45fad83d478eac535a6d435ed165', 'request_id': 'req-7f522b16-1208-4698-b988-4868830644c5'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:31.743720 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-7f522b16-1208-4698-b988-4868830644c5 admin admin] The token cache is being invalidate because project 8cfa45fad83d478eac535a6d435ed165 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:16:31.746392 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-7f522b16-1208-4698-b988-4868830644c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:31.746392 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-7f522b16-1208-4698-b988-4868830644c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:31.748753 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1349/2694] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:16:31 2022] DELETE /identity/v3/projects/8cfa45fad83d478eac535a6d435ed165 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:16:31.754002 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f522b16-1208-4698-b988-4868830644c5 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:31.755657 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-7f522b16-1208-4698-b988-4868830644c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:16:31.778398 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f084626b-9b36-4172-9220-43e25d823028 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:31.833910 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f522b16-1208-4698-b988-4868830644c5 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:31.840996 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717fcc49-cf73-442f-b60b-a0483843325a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:31.841682 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fcc49-cf73-442f-b60b-a0483843325a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:31.842110 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fcc49-cf73-442f-b60b-a0483843325a admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:31.842292 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fcc49-cf73-442f-b60b-a0483843325a admin admin] PATH_INFO: `/v3/users/2682e742621643ae9b210c809ff5fafc` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:31.843613 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717fcc49-cf73-442f-b60b-a0483843325a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2682e742621643ae9b210c809ff5fafc)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:31.845640 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717fcc49-cf73-442f-b60b-a0483843325a admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:31.858024 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ca025-d557-4d6b-88f2-f0ae4b645571 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:31.858652 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ca025-d557-4d6b-88f2-f0ae4b645571 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:31.859196 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ca025-d557-4d6b-88f2-f0ae4b645571 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:31.859196 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ca025-d557-4d6b-88f2-f0ae4b645571 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:31.861037 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d90ca025-d557-4d6b-88f2-f0ae4b645571 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:16:31.881668 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:16:31.926953 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90ca025-d557-4d6b-88f2-f0ae4b645571 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:31.928490 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90ca025-d557-4d6b-88f2-f0ae4b645571 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:31.938371 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1346/2695] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:16:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 01:16:32.075470 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-717fcc49-cf73-442f-b60b-a0483843325a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2682e742621643ae9b210c809ff5fafc'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:32.078573 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-717fcc49-cf73-442f-b60b-a0483843325a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2682e742621643ae9b210c809ff5fafc'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:32.167238 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-717fcc49-cf73-442f-b60b-a0483843325a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2682e742621643ae9b210c809ff5fafc'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:32.168072 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-717fcc49-cf73-442f-b60b-a0483843325a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2682e742621643ae9b210c809ff5fafc'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:32.188706 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-717fcc49-cf73-442f-b60b-a0483843325a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2682e742621643ae9b210c809ff5fafc'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:32.204301 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1350/2696] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:16:31 2022] DELETE /identity/v3/users/2682e742621643ae9b210c809ff5fafc => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:16:32.649040 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ca025-d557-4d6b-88f2-f0ae4b645571 service nova] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:32.665951 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ca025-d557-4d6b-88f2-f0ae4b645571 service nova] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:32.732269 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f680cedb-0042-4820-8267-15231aef4acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:32.733142 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680cedb-0042-4820-8267-15231aef4acd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:32.733342 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680cedb-0042-4820-8267-15231aef4acd admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:32.733829 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680cedb-0042-4820-8267-15231aef4acd admin admin] PATH_INFO: `/v3/projects/8d5910cdc44f4de69c9b238bf8e80698` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:32.735080 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f680cedb-0042-4820-8267-15231aef4acd admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d5910cdc44f4de69c9b238bf8e80698)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:32.737038 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f680cedb-0042-4820-8267-15231aef4acd admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:32.854854 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f680cedb-0042-4820-8267-15231aef4acd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d5910cdc44f4de69c9b238bf8e80698', 'request_id': 'req-f680cedb-0042-4820-8267-15231aef4acd'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:32.922501 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f680cedb-0042-4820-8267-15231aef4acd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d5910cdc44f4de69c9b238bf8e80698', 'request_id': 'req-f680cedb-0042-4820-8267-15231aef4acd'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:32.929892 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f680cedb-0042-4820-8267-15231aef4acd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d5910cdc44f4de69c9b238bf8e80698', 'request_id': 'req-f680cedb-0042-4820-8267-15231aef4acd'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:32.943042 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f680cedb-0042-4820-8267-15231aef4acd admin admin] The token cache is being invalidate because project 8d5910cdc44f4de69c9b238bf8e80698 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:16:32.943339 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f680cedb-0042-4820-8267-15231aef4acd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:32.944000 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f680cedb-0042-4820-8267-15231aef4acd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:32.945838 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1347/2697] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:16:32 2022] DELETE /identity/v3/projects/8d5910cdc44f4de69c9b238bf8e80698 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:16:32.951710 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb51a1a-ad36-4c2f-ad82-c16ee789f032 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:32.952216 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb51a1a-ad36-4c2f-ad82-c16ee789f032 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:32.952340 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb51a1a-ad36-4c2f-ad82-c16ee789f032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:32.961519 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:16:32.973830 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-edb51a1a-ad36-4c2f-ad82-c16ee789f032 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:16:32.975101 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-edb51a1a-ad36-4c2f-ad82-c16ee789f032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:16:33.041456 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1351/2698] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:16:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:16:33.050977 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf605d71-c2ea-4a2b-8b99-8ebd37e5fe83 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:33.051627 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf605d71-c2ea-4a2b-8b99-8ebd37e5fe83 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:33.051732 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf605d71-c2ea-4a2b-8b99-8ebd37e5fe83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:33.073557 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-bf605d71-c2ea-4a2b-8b99-8ebd37e5fe83 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:16:33.074806 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bf605d71-c2ea-4a2b-8b99-8ebd37e5fe83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:16:33.083345 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1348/2699] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:16:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:16:33.088041 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb51a1a-ad36-4c2f-ad82-c16ee789f032 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:33.098424 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb51a1a-ad36-4c2f-ad82-c16ee789f032 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:33.104132 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b7cf32-4790-4d7a-a4a7-b6057e1cd97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:33.104801 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7cf32-4790-4d7a-a4a7-b6057e1cd97e admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:33.105080 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7cf32-4790-4d7a-a4a7-b6057e1cd97e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:33.105352 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7cf32-4790-4d7a-a4a7-b6057e1cd97e admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:33.106069 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b7cf32-4790-4d7a-a4a7-b6057e1cd97e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:16:33.106372 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b7cf32-4790-4d7a-a4a7-b6057e1cd97e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:33.108010 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b7cf32-4790-4d7a-a4a7-b6057e1cd97e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:33.125320 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1352/2700] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:16:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 01:16:33.127497 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf605d71-c2ea-4a2b-8b99-8ebd37e5fe83 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:33.139075 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf605d71-c2ea-4a2b-8b99-8ebd37e5fe83 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:33.144376 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156a85d-0482-4e30-b708-d2e506e06337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:33.145079 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156a85d-0482-4e30-b708-d2e506e06337 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:33.145355 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156a85d-0482-4e30-b708-d2e506e06337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:33.145623 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156a85d-0482-4e30-b708-d2e506e06337 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:33.146264 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156a85d-0482-4e30-b708-d2e506e06337 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:33.148915 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156a85d-0482-4e30-b708-d2e506e06337 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:33.243424 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1349/2701] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:16:33 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:16:33.247875 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b7cf32-4790-4d7a-a4a7-b6057e1cd97e admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:33.259055 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b7cf32-4790-4d7a-a4a7-b6057e1cd97e admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:33.318958 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a8f406-ce7c-4866-9acc-1e4bd90dcab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:33.319607 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8f406-ce7c-4866-9acc-1e4bd90dcab7 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:33.319880 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8f406-ce7c-4866-9acc-1e4bd90dcab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:33.320201 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8f406-ce7c-4866-9acc-1e4bd90dcab7 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:33.320900 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a8f406-ce7c-4866-9acc-1e4bd90dcab7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:33.322813 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a8f406-ce7c-4866-9acc-1e4bd90dcab7 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:33.433831 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1353/2702] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:16:33 2022] POST /identity/v3/users => generated 289 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:16:33.436029 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156a85d-0482-4e30-b708-d2e506e06337 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:33.447125 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156a85d-0482-4e30-b708-d2e506e06337 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:33.452758 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d53eb-6638-41e9-9393-4f1b5091758c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:33.453382 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d53eb-6638-41e9-9393-4f1b5091758c admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:33.453668 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d53eb-6638-41e9-9393-4f1b5091758c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:33.453947 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d53eb-6638-41e9-9393-4f1b5091758c admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:33.454594 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94d53eb-6638-41e9-9393-4f1b5091758c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:33.456205 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94d53eb-6638-41e9-9393-4f1b5091758c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:33.466695 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1350/2703] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 01:16:33.470893 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a8f406-ce7c-4866-9acc-1e4bd90dcab7 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:33.483645 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a8f406-ce7c-4866-9acc-1e4bd90dcab7 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:33.490007 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840305ab-a15e-4f2e-b9a5-1f60f2c67b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:33.490641 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840305ab-a15e-4f2e-b9a5-1f60f2c67b5a admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:33.490927 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840305ab-a15e-4f2e-b9a5-1f60f2c67b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:33.491294 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840305ab-a15e-4f2e-b9a5-1f60f2c67b5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:33.492498 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840305ab-a15e-4f2e-b9a5-1f60f2c67b5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:33.494110 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840305ab-a15e-4f2e-b9a5-1f60f2c67b5a admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:33.510662 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1354/2704] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:16:33.511331 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d53eb-6638-41e9-9393-4f1b5091758c admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:33.527807 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d53eb-6638-41e9-9393-4f1b5091758c admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:33.532957 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d601ecaa-cae3-4c20-b9a6-e32ecf3fed1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:33.533630 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601ecaa-cae3-4c20-b9a6-e32ecf3fed1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:33.533915 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601ecaa-cae3-4c20-b9a6-e32ecf3fed1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:33.534178 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601ecaa-cae3-4c20-b9a6-e32ecf3fed1e admin admin] PATH_INFO: `/v3/projects/8d9dbcf905c742a1b875411d252c4846/users/2327c91cd57845f9bbc0aecf9e2f462b/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:33.537772 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:16:33.558354 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d601ecaa-cae3-4c20-b9a6-e32ecf3fed1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d9dbcf905c742a1b875411d252c4846, user_id=2327c91cd57845f9bbc0aecf9e2f462b, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:33.561765 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d601ecaa-cae3-4c20-b9a6-e32ecf3fed1e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:33.665188 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1351/2705] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:16:33 2022] PUT /identity/v3/projects/8d9dbcf905c742a1b875411d252c4846/users/2327c91cd57845f9bbc0aecf9e2f462b/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:16:33.675763 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c8aa1-4140-4809-b8cb-2ea9b6d36896 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:33.675991 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c8aa1-4140-4809-b8cb-2ea9b6d36896 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:33.676272 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c8aa1-4140-4809-b8cb-2ea9b6d36896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:33.721084 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-202c8aa1-4140-4809-b8cb-2ea9b6d36896 None None] MFA Rules not processed for user `2327c91cd57845f9bbc0aecf9e2f462b`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:16:33.721084 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-202c8aa1-4140-4809-b8cb-2ea9b6d36896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:16:33.790593 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:16:33.795052 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1355/2706] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:16:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:16:33.808407 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d601ecaa-cae3-4c20-b9a6-e32ecf3fed1e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:33.817515 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d601ecaa-cae3-4c20-b9a6-e32ecf3fed1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:16:33.878469 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d601ecaa-cae3-4c20-b9a6-e32ecf3fed1e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:33.882316 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9832b-5dfa-48a1-a483-82249545f08b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:33.882938 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9832b-5dfa-48a1-a483-82249545f08b service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:33.883220 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9832b-5dfa-48a1-a483-82249545f08b service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:33.883495 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9832b-5dfa-48a1-a483-82249545f08b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:33.893630 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d9832b-5dfa-48a1-a483-82249545f08b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:33.895076 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d9832b-5dfa-48a1-a483-82249545f08b service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:33.931510 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1352/2707] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:16:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:16:34.573872 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bbe2e-8b36-453c-a0c7-4e8e9398fa22 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:34.574413 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bbe2e-8b36-453c-a0c7-4e8e9398fa22 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:34.574541 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bbe2e-8b36-453c-a0c7-4e8e9398fa22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:34.593766 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-f43bbe2e-8b36-453c-a0c7-4e8e9398fa22 None None] MFA Rules not processed for user `c5819e21fbba41d3b235daf8f0d6769b`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:16:34.598529 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f43bbe2e-8b36-453c-a0c7-4e8e9398fa22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:16:34.674363 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1356/2708] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:16:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:16:34.723987 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9832b-5dfa-48a1-a483-82249545f08b service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:34.726379 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-49d9832b-5dfa-48a1-a483-82249545f08b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:16:34.788280 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9832b-5dfa-48a1-a483-82249545f08b service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:34.791766 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c2f09a-267f-4a56-9462-4f3995b85945 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1179503bdab547c99175a27b7ce27b9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:34.792434 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2f09a-267f-4a56-9462-4f3995b85945 service glance] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:34.792742 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2f09a-267f-4a56-9462-4f3995b85945 service glance] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:34.793013 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2f09a-267f-4a56-9462-4f3995b85945 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:34.794744 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-29c2f09a-267f-4a56-9462-4f3995b85945 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:16:34.873287 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c2f09a-267f-4a56-9462-4f3995b85945 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:34.874661 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c2f09a-267f-4a56-9462-4f3995b85945 service glance] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:34.888304 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1353/2709] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:16:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:16:35.632274 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43bbe2e-8b36-453c-a0c7-4e8e9398fa22 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:35.634843 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f43bbe2e-8b36-453c-a0c7-4e8e9398fa22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:16:35.709503 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43bbe2e-8b36-453c-a0c7-4e8e9398fa22 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:35.713123 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5f1b3-383c-452d-83df-5b991ea7f2cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:35.714079 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5f1b3-383c-452d-83df-5b991ea7f2cd service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:35.714509 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5f1b3-383c-452d-83df-5b991ea7f2cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:35.715082 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5f1b3-383c-452d-83df-5b991ea7f2cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:35.719070 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-afd5f1b3-383c-452d-83df-5b991ea7f2cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:16:35.742175 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd5f1b3-383c-452d-83df-5b991ea7f2cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:35.745035 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd5f1b3-383c-452d-83df-5b991ea7f2cd service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:35.776197 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1357/2710] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:16:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:16:44.881509 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=86,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:16:45.756300 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=51 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:16:51.078790 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c2f09a-267f-4a56-9462-4f3995b85945 service glance] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:51.143514 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c2f09a-267f-4a56-9462-4f3995b85945 service glance] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:51.147969 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b10c24a-695a-45e2-a5da-a614e5d4446f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:51.148649 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10c24a-695a-45e2-a5da-a614e5d4446f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:51.148928 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10c24a-695a-45e2-a5da-a614e5d4446f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:51.149205 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10c24a-695a-45e2-a5da-a614e5d4446f admin admin] PATH_INFO: `/v3/users/2327c91cd57845f9bbc0aecf9e2f462b` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:51.150302 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b10c24a-695a-45e2-a5da-a614e5d4446f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2327c91cd57845f9bbc0aecf9e2f462b)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:51.152182 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b10c24a-695a-45e2-a5da-a614e5d4446f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:51.306119 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8b10c24a-695a-45e2-a5da-a614e5d4446f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2327c91cd57845f9bbc0aecf9e2f462b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:51.306765 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8b10c24a-695a-45e2-a5da-a614e5d4446f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2327c91cd57845f9bbc0aecf9e2f462b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:51.313642 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8b10c24a-695a-45e2-a5da-a614e5d4446f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2327c91cd57845f9bbc0aecf9e2f462b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:51.388661 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8b10c24a-695a-45e2-a5da-a614e5d4446f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2327c91cd57845f9bbc0aecf9e2f462b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:51.389231 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8b10c24a-695a-45e2-a5da-a614e5d4446f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2327c91cd57845f9bbc0aecf9e2f462b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:51.406770 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1354/2711] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:16:51 2022] DELETE /identity/v3/users/2327c91cd57845f9bbc0aecf9e2f462b => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:16:51.411243 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e562e73-fc30-4a83-8e42-6ce178574ff2 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:51.415371 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e562e73-fc30-4a83-8e42-6ce178574ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:51.415371 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e562e73-fc30-4a83-8e42-6ce178574ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:51.431529 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-3e562e73-fc30-4a83-8e42-6ce178574ff2 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:16:51.432766 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3e562e73-fc30-4a83-8e42-6ce178574ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:16:51.490402 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1358/2712] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:16:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:16:51.501081 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b10c24a-695a-45e2-a5da-a614e5d4446f admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:51.502482 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8b10c24a-695a-45e2-a5da-a614e5d4446f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:16:51.560382 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b10c24a-695a-45e2-a5da-a614e5d4446f admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:51.564058 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d14acd-a668-4a5a-ad20-27d185bf0ae5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:51.564687 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d14acd-a668-4a5a-ad20-27d185bf0ae5 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:51.564966 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d14acd-a668-4a5a-ad20-27d185bf0ae5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:51.565265 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d14acd-a668-4a5a-ad20-27d185bf0ae5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:51.574647 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d14acd-a668-4a5a-ad20-27d185bf0ae5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:51.576077 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d14acd-a668-4a5a-ad20-27d185bf0ae5 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:51.585891 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1355/2713] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:16:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:16:51.969344 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e562e73-fc30-4a83-8e42-6ce178574ff2 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:51.973012 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3e562e73-fc30-4a83-8e42-6ce178574ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:16:51.982946 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e562e73-fc30-4a83-8e42-6ce178574ff2 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:51.988079 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e5c11-9f7b-43e8-a55d-9e75fb887c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:51.988728 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e5c11-9f7b-43e8-a55d-9e75fb887c55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:51.989005 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e5c11-9f7b-43e8-a55d-9e75fb887c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:51.989268 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e5c11-9f7b-43e8-a55d-9e75fb887c55 admin admin] PATH_INFO: `/v3/projects/8d9dbcf905c742a1b875411d252c4846` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:51.990374 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e5c11-9f7b-43e8-a55d-9e75fb887c55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d9dbcf905c742a1b875411d252c4846)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:51.992532 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e5c11-9f7b-43e8-a55d-9e75fb887c55 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:52.076005 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-dc6e5c11-9f7b-43e8-a55d-9e75fb887c55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d9dbcf905c742a1b875411d252c4846', 'request_id': 'req-dc6e5c11-9f7b-43e8-a55d-9e75fb887c55'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:52.133184 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-dc6e5c11-9f7b-43e8-a55d-9e75fb887c55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d9dbcf905c742a1b875411d252c4846', 'request_id': 'req-dc6e5c11-9f7b-43e8-a55d-9e75fb887c55'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:52.139255 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-dc6e5c11-9f7b-43e8-a55d-9e75fb887c55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d9dbcf905c742a1b875411d252c4846', 'request_id': 'req-dc6e5c11-9f7b-43e8-a55d-9e75fb887c55'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:52.151154 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-dc6e5c11-9f7b-43e8-a55d-9e75fb887c55 admin admin] The token cache is being invalidate because project 8d9dbcf905c742a1b875411d252c4846 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:16:52.151446 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-dc6e5c11-9f7b-43e8-a55d-9e75fb887c55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:52.152075 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-dc6e5c11-9f7b-43e8-a55d-9e75fb887c55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:16:52.153578 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1359/2714] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:16:51 2022] DELETE /identity/v3/projects/8d9dbcf905c742a1b875411d252c4846 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:16:52.158572 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94e8d9-deea-4c6b-bf53-b4e5958dc188 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:52.159041 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94e8d9-deea-4c6b-bf53-b4e5958dc188 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:52.159138 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94e8d9-deea-4c6b-bf53-b4e5958dc188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:52.179701 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1d94e8d9-deea-4c6b-bf53-b4e5958dc188 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:16:52.180929 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1d94e8d9-deea-4c6b-bf53-b4e5958dc188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:16:52.197657 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:16:52.240670 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1356/2715] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:16:52.249146 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8987ff5-51a1-4dcd-a3c8-6281e3b77626 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:52.249433 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8987ff5-51a1-4dcd-a3c8-6281e3b77626 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:52.249821 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8987ff5-51a1-4dcd-a3c8-6281e3b77626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:52.270114 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-c8987ff5-51a1-4dcd-a3c8-6281e3b77626 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:16:52.271298 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c8987ff5-51a1-4dcd-a3c8-6281e3b77626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:16:52.279057 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1360/2716] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:16:52.283312 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d94e8d9-deea-4c6b-bf53-b4e5958dc188 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:52.293187 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d94e8d9-deea-4c6b-bf53-b4e5958dc188 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:52.297739 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f55cfb-ddb7-40be-ba37-742bb246404c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:52.298328 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f55cfb-ddb7-40be-ba37-742bb246404c admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:52.300690 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f55cfb-ddb7-40be-ba37-742bb246404c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:52.300690 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f55cfb-ddb7-40be-ba37-742bb246404c admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:52.300690 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f55cfb-ddb7-40be-ba37-742bb246404c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:16:52.300690 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f55cfb-ddb7-40be-ba37-742bb246404c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:52.301414 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f55cfb-ddb7-40be-ba37-742bb246404c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:52.313509 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1357/2717] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:16:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 01:16:52.317795 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8987ff5-51a1-4dcd-a3c8-6281e3b77626 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:52.327407 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8987ff5-51a1-4dcd-a3c8-6281e3b77626 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:52.332182 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2596ac59-f559-4d78-90c3-c50836053584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:52.332797 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2596ac59-f559-4d78-90c3-c50836053584 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:52.333071 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2596ac59-f559-4d78-90c3-c50836053584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:52.333342 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2596ac59-f559-4d78-90c3-c50836053584 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:52.333957 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2596ac59-f559-4d78-90c3-c50836053584 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:52.336073 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2596ac59-f559-4d78-90c3-c50836053584 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:52.413952 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1361/2718] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:16:52 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:16:52.418181 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f55cfb-ddb7-40be-ba37-742bb246404c admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:52.428269 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f55cfb-ddb7-40be-ba37-742bb246404c admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:52.484440 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3551b898-3e50-4ad5-90d5-b648b93f8010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:52.484987 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551b898-3e50-4ad5-90d5-b648b93f8010 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:52.485265 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551b898-3e50-4ad5-90d5-b648b93f8010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:52.485536 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551b898-3e50-4ad5-90d5-b648b93f8010 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:52.486260 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3551b898-3e50-4ad5-90d5-b648b93f8010 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:52.488138 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3551b898-3e50-4ad5-90d5-b648b93f8010 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:52.584270 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1358/2719] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:16:52 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:16:52.589185 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2596ac59-f559-4d78-90c3-c50836053584 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:52.600854 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2596ac59-f559-4d78-90c3-c50836053584 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:52.605630 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f33d8-c186-4fa0-9cc4-39daabcc43e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:52.606232 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f33d8-c186-4fa0-9cc4-39daabcc43e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:52.606511 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f33d8-c186-4fa0-9cc4-39daabcc43e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:52.606787 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f33d8-c186-4fa0-9cc4-39daabcc43e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:52.607424 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74f33d8-c186-4fa0-9cc4-39daabcc43e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:52.609044 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74f33d8-c186-4fa0-9cc4-39daabcc43e5 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:52.619633 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1362/2720] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:16:52.623768 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3551b898-3e50-4ad5-90d5-b648b93f8010 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:52.634340 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3551b898-3e50-4ad5-90d5-b648b93f8010 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:52.638898 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a0c5a-5c45-41ff-a0a6-9066052a9a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:52.639483 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a0c5a-5c45-41ff-a0a6-9066052a9a23 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:52.642748 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a0c5a-5c45-41ff-a0a6-9066052a9a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:52.642748 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a0c5a-5c45-41ff-a0a6-9066052a9a23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:52.642748 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217a0c5a-5c45-41ff-a0a6-9066052a9a23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:52.642748 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217a0c5a-5c45-41ff-a0a6-9066052a9a23 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:52.653038 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1359/2721] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 01:16:52.657114 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f33d8-c186-4fa0-9cc4-39daabcc43e5 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:52.667702 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f33d8-c186-4fa0-9cc4-39daabcc43e5 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:52.672371 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe240f7-6037-4468-9b78-fb325e324c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:52.673033 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe240f7-6037-4468-9b78-fb325e324c03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:52.673311 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe240f7-6037-4468-9b78-fb325e324c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:52.673583 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe240f7-6037-4468-9b78-fb325e324c03 admin admin] PATH_INFO: `/v3/projects/28c37a44130b4581841a5d0d7e52025d/users/85695d4f33f44e1bb696a6b12083a149/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:52.688169 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe240f7-6037-4468-9b78-fb325e324c03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28c37a44130b4581841a5d0d7e52025d, user_id=85695d4f33f44e1bb696a6b12083a149, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:52.690862 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe240f7-6037-4468-9b78-fb325e324c03 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:52.737787 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1363/2722] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:16:52 2022] PUT /identity/v3/projects/28c37a44130b4581841a5d0d7e52025d/users/85695d4f33f44e1bb696a6b12083a149/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:16:52.749576 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7172f5-3b7f-43c9-bd57-026e5dc0b893 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:52.749865 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7172f5-3b7f-43c9-bd57-026e5dc0b893 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:52.750139 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7172f5-3b7f-43c9-bd57-026e5dc0b893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:52.777500 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-3a7172f5-3b7f-43c9-bd57-026e5dc0b893 None None] MFA Rules not processed for user `85695d4f33f44e1bb696a6b12083a149`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:16:52.778708 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3a7172f5-3b7f-43c9-bd57-026e5dc0b893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:16:52.842496 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1360/2723] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:16:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:16:52.847024 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe240f7-6037-4468-9b78-fb325e324c03 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:52.868540 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:16:52.913361 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe240f7-6037-4468-9b78-fb325e324c03 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:52.917932 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e8cd7-7d56-4d14-8609-75ac336e88f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:52.918604 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e8cd7-7d56-4d14-8609-75ac336e88f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:52.918883 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e8cd7-7d56-4d14-8609-75ac336e88f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:52.919155 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e8cd7-7d56-4d14-8609-75ac336e88f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:52.919786 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e8cd7-7d56-4d14-8609-75ac336e88f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:52.921702 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e8cd7-7d56-4d14-8609-75ac336e88f6 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:53.018201 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1364/2724] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:16:52 2022] POST /identity/v3/projects => generated 383 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:16:53.030355 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7172f5-3b7f-43c9-bd57-026e5dc0b893 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:53.042000 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7172f5-3b7f-43c9-bd57-026e5dc0b893 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:53.054460 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:16:53.098123 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a02fcb-0afb-4b5e-bfe6-2353926f928a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:53.098856 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a02fcb-0afb-4b5e-bfe6-2353926f928a admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:53.099580 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a02fcb-0afb-4b5e-bfe6-2353926f928a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:53.099839 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a02fcb-0afb-4b5e-bfe6-2353926f928a admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:53.101657 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a02fcb-0afb-4b5e-bfe6-2353926f928a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:53.103557 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a02fcb-0afb-4b5e-bfe6-2353926f928a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:53.195532 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1361/2725] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:16:53 2022] POST /identity/v3/users => generated 292 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:16:53.199839 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e8cd7-7d56-4d14-8609-75ac336e88f6 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:53.210573 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e8cd7-7d56-4d14-8609-75ac336e88f6 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:53.215174 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cdbb5a-704a-4e8f-b19e-372c64348c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:53.215761 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cdbb5a-704a-4e8f-b19e-372c64348c9d admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:53.216039 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cdbb5a-704a-4e8f-b19e-372c64348c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:53.216310 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cdbb5a-704a-4e8f-b19e-372c64348c9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:53.217112 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cdbb5a-704a-4e8f-b19e-372c64348c9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:53.218698 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cdbb5a-704a-4e8f-b19e-372c64348c9d admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:53.230269 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1365/2726] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:16:53.234078 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a02fcb-0afb-4b5e-bfe6-2353926f928a admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:53.244349 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a02fcb-0afb-4b5e-bfe6-2353926f928a admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:53.248914 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a81b07e-bd71-42ad-9b3c-d946df963161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:53.249505 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a81b07e-bd71-42ad-9b3c-d946df963161 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:53.249789 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a81b07e-bd71-42ad-9b3c-d946df963161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:53.250059 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a81b07e-bd71-42ad-9b3c-d946df963161 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:53.250688 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a81b07e-bd71-42ad-9b3c-d946df963161 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:53.252262 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a81b07e-bd71-42ad-9b3c-d946df963161 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:53.262591 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1362/2727] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:16:53.266530 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cdbb5a-704a-4e8f-b19e-372c64348c9d admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:53.277076 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cdbb5a-704a-4e8f-b19e-372c64348c9d admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:53.281579 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7e5fe-3af4-4ddb-aeaf-90bc16ae6096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:53.282216 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7e5fe-3af4-4ddb-aeaf-90bc16ae6096 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:53.282498 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7e5fe-3af4-4ddb-aeaf-90bc16ae6096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:53.282773 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7e5fe-3af4-4ddb-aeaf-90bc16ae6096 admin admin] PATH_INFO: `/v3/projects/0d5becbfbc444d9483489251e59ce15c/users/d0850f597f784be99cdae731c3d0e5cd/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:53.297319 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf7e5fe-3af4-4ddb-aeaf-90bc16ae6096 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d5becbfbc444d9483489251e59ce15c, user_id=d0850f597f784be99cdae731c3d0e5cd, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:53.299937 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf7e5fe-3af4-4ddb-aeaf-90bc16ae6096 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:53.385240 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1366/2728] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:16:53 2022] PUT /identity/v3/projects/0d5becbfbc444d9483489251e59ce15c/users/d0850f597f784be99cdae731c3d0e5cd/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:16:53.395539 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0400e7-0351-467e-9f83-8e92e89955bd None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:53.395831 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0400e7-0351-467e-9f83-8e92e89955bd None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:53.396112 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0400e7-0351-467e-9f83-8e92e89955bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:53.424730 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-ca0400e7-0351-467e-9f83-8e92e89955bd None None] MFA Rules not processed for user `d0850f597f784be99cdae731c3d0e5cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:16:53.425951 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ca0400e7-0351-467e-9f83-8e92e89955bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:16:53.490595 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1363/2729] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:16:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:16:53.495250 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7e5fe-3af4-4ddb-aeaf-90bc16ae6096 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:53.556807 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7e5fe-3af4-4ddb-aeaf-90bc16ae6096 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:53.561346 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f1b5b-6318-4335-93fe-b113fd17512e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:53.561965 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f1b5b-6318-4335-93fe-b113fd17512e admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:53.562242 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f1b5b-6318-4335-93fe-b113fd17512e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:53.562512 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f1b5b-6318-4335-93fe-b113fd17512e admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:53.563161 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f1b5b-6318-4335-93fe-b113fd17512e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:53.565075 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f1b5b-6318-4335-93fe-b113fd17512e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:53.659994 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1367/2730] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:16:53 2022] POST /identity/v3/projects => generated 385 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:16:53.664803 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0400e7-0351-467e-9f83-8e92e89955bd None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:53.675441 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0400e7-0351-467e-9f83-8e92e89955bd None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:53.731992 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b67b638-9186-4a85-9a33-ada44b5b463c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:53.732629 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67b638-9186-4a85-9a33-ada44b5b463c admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:53.732908 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67b638-9186-4a85-9a33-ada44b5b463c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:53.733178 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67b638-9186-4a85-9a33-ada44b5b463c admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:53.733825 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b67b638-9186-4a85-9a33-ada44b5b463c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:53.735744 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b67b638-9186-4a85-9a33-ada44b5b463c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:53.807790 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1364/2731] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:16:53 2022] POST /identity/v3/users => generated 299 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:16:53.811873 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f1b5b-6318-4335-93fe-b113fd17512e admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:53.822739 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f1b5b-6318-4335-93fe-b113fd17512e admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:53.827346 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b902fe-1802-4e07-8b15-ede9fb8eadab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:53.827939 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b902fe-1802-4e07-8b15-ede9fb8eadab admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:53.828300 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b902fe-1802-4e07-8b15-ede9fb8eadab admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:53.828608 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b902fe-1802-4e07-8b15-ede9fb8eadab admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:53.829240 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b902fe-1802-4e07-8b15-ede9fb8eadab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:53.830821 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b902fe-1802-4e07-8b15-ede9fb8eadab admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:53.841967 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1368/2732] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:16:53.848778 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b67b638-9186-4a85-9a33-ada44b5b463c admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:53.858984 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b67b638-9186-4a85-9a33-ada44b5b463c admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:53.863677 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81c2bbd-9fe8-48ef-8a9e-7195198389ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:53.864321 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c2bbd-9fe8-48ef-8a9e-7195198389ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:53.864632 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c2bbd-9fe8-48ef-8a9e-7195198389ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:53.864909 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c2bbd-9fe8-48ef-8a9e-7195198389ee admin admin] PATH_INFO: `/v3/projects/bb2f3de345d2454d838eb5ab953b6604/users/ea1f3f870c4944f3ae2cc38f00f8a116/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:53.878003 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:16:53.881328 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81c2bbd-9fe8-48ef-8a9e-7195198389ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb2f3de345d2454d838eb5ab953b6604, user_id=ea1f3f870c4944f3ae2cc38f00f8a116, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:53.885101 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81c2bbd-9fe8-48ef-8a9e-7195198389ee admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:53.978282 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1365/2733] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:16:53 2022] PUT /identity/v3/projects/bb2f3de345d2454d838eb5ab953b6604/users/ea1f3f870c4944f3ae2cc38f00f8a116/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:16:53.982185 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b902fe-1802-4e07-8b15-ede9fb8eadab admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:53.992661 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b902fe-1802-4e07-8b15-ede9fb8eadab admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:54.033218 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:16:54.047360 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ca56f-1530-422d-82ad-b965b3e1ce91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:54.047955 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ca56f-1530-422d-82ad-b965b3e1ce91 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:54.048232 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ca56f-1530-422d-82ad-b965b3e1ce91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:54.048526 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ca56f-1530-422d-82ad-b965b3e1ce91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:54.049153 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0ca56f-1530-422d-82ad-b965b3e1ce91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:54.050729 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0ca56f-1530-422d-82ad-b965b3e1ce91 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:54.060564 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1369/2734] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:16:54.064667 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81c2bbd-9fe8-48ef-8a9e-7195198389ee admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:54.075518 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81c2bbd-9fe8-48ef-8a9e-7195198389ee admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:54.080428 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa3dd8c-1415-41e8-993d-e27b40ae012e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:54.081118 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3dd8c-1415-41e8-993d-e27b40ae012e admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:54.081399 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3dd8c-1415-41e8-993d-e27b40ae012e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:54.081674 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3dd8c-1415-41e8-993d-e27b40ae012e admin admin] PATH_INFO: `/v3/projects/bb2f3de345d2454d838eb5ab953b6604/users/ea1f3f870c4944f3ae2cc38f00f8a116/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:54.083682 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa3dd8c-1415-41e8-993d-e27b40ae012e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb2f3de345d2454d838eb5ab953b6604, user_id=ea1f3f870c4944f3ae2cc38f00f8a116, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:54.086901 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa3dd8c-1415-41e8-993d-e27b40ae012e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:54.097150 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1366/2735] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:16:54 2022] PUT /identity/v3/projects/bb2f3de345d2454d838eb5ab953b6604/users/ea1f3f870c4944f3ae2cc38f00f8a116/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:16:54.109098 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3466bfdd-d841-4f48-86b2-8d68b94d5102 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:54.109385 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3466bfdd-d841-4f48-86b2-8d68b94d5102 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:54.109662 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3466bfdd-d841-4f48-86b2-8d68b94d5102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:54.138065 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-3466bfdd-d841-4f48-86b2-8d68b94d5102 None None] MFA Rules not processed for user `ea1f3f870c4944f3ae2cc38f00f8a116`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:16:54.139272 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3466bfdd-d841-4f48-86b2-8d68b94d5102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:16:54.207196 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1370/2736] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:16:54 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:16:54.218840 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa3dd8c-1415-41e8-993d-e27b40ae012e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:54.220578 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-cfa3dd8c-1415-41e8-993d-e27b40ae012e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:16:54.278917 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa3dd8c-1415-41e8-993d-e27b40ae012e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:54.282556 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d15f0-9b60-4fec-9dc9-52c077d2534f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:54.283154 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d15f0-9b60-4fec-9dc9-52c077d2534f service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:54.283429 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d15f0-9b60-4fec-9dc9-52c077d2534f service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:54.283699 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d15f0-9b60-4fec-9dc9-52c077d2534f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:54.340390 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484d15f0-9b60-4fec-9dc9-52c077d2534f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:54.341772 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484d15f0-9b60-4fec-9dc9-52c077d2534f service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:54.351151 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1367/2737] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:16:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:16:56.297774 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3466bfdd-d841-4f48-86b2-8d68b94d5102 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:56.299253 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3466bfdd-d841-4f48-86b2-8d68b94d5102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:16:56.311416 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3466bfdd-d841-4f48-86b2-8d68b94d5102 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:56.315944 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76fea2-7fbb-40f5-a9d9-9e27ad162d33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:56.318224 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76fea2-7fbb-40f5-a9d9-9e27ad162d33 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:56.318224 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76fea2-7fbb-40f5-a9d9-9e27ad162d33 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:56.318224 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76fea2-7fbb-40f5-a9d9-9e27ad162d33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:56.376904 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f76fea2-7fbb-40f5-a9d9-9e27ad162d33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:56.378442 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f76fea2-7fbb-40f5-a9d9-9e27ad162d33 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:56.389620 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1371/2738] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:16:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 01:16:57.563954 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d15f0-9b60-4fec-9dc9-52c077d2534f service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:16:57.572633 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d15f0-9b60-4fec-9dc9-52c077d2534f service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:16:57.576172 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07186e-ce91-4ba9-8b6e-32d0233d1ced service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:16:57.576858 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07186e-ce91-4ba9-8b6e-32d0233d1ced service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:16:57.577136 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07186e-ce91-4ba9-8b6e-32d0233d1ced service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:16:57.577405 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07186e-ce91-4ba9-8b6e-32d0233d1ced service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:16:57.586873 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07186e-ce91-4ba9-8b6e-32d0233d1ced service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:16:57.588285 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07186e-ce91-4ba9-8b6e-32d0233d1ced service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:16:57.597839 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1368/2739] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:16:57 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:17:02.042571 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76fea2-7fbb-40f5-a9d9-9e27ad162d33 service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:02.105919 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76fea2-7fbb-40f5-a9d9-9e27ad162d33 service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:02.110492 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22585cd3-0041-4662-85fc-bf07a71e4d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:02.111092 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22585cd3-0041-4662-85fc-bf07a71e4d80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:02.111371 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22585cd3-0041-4662-85fc-bf07a71e4d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:02.111643 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22585cd3-0041-4662-85fc-bf07a71e4d80 admin admin] PATH_INFO: `/v3/users/85695d4f33f44e1bb696a6b12083a149` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:02.112788 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22585cd3-0041-4662-85fc-bf07a71e4d80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85695d4f33f44e1bb696a6b12083a149)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:02.114693 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22585cd3-0041-4662-85fc-bf07a71e4d80 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:02.119513 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=100 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:17:02.271829 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-22585cd3-0041-4662-85fc-bf07a71e4d80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85695d4f33f44e1bb696a6b12083a149'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:02.272424 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-22585cd3-0041-4662-85fc-bf07a71e4d80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85695d4f33f44e1bb696a6b12083a149'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:02.321551 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-22585cd3-0041-4662-85fc-bf07a71e4d80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85695d4f33f44e1bb696a6b12083a149'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:02.322152 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-22585cd3-0041-4662-85fc-bf07a71e4d80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85695d4f33f44e1bb696a6b12083a149'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:02.333836 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-22585cd3-0041-4662-85fc-bf07a71e4d80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85695d4f33f44e1bb696a6b12083a149'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:02.346374 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1372/2740] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:17:02 2022] DELETE /identity/v3/users/85695d4f33f44e1bb696a6b12083a149 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:17:02.350987 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7cfe4-8d08-4c85-a86c-ee89df16abbb None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:02.351477 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7cfe4-8d08-4c85-a86c-ee89df16abbb None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:02.351586 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7cfe4-8d08-4c85-a86c-ee89df16abbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:02.371341 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-2cd7cfe4-8d08-4c85-a86c-ee89df16abbb None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:17:02.372681 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2cd7cfe4-8d08-4c85-a86c-ee89df16abbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:17:02.429696 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1369/2741] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:17:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:17:02.441271 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22585cd3-0041-4662-85fc-bf07a71e4d80 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:02.443308 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-22585cd3-0041-4662-85fc-bf07a71e4d80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:17:02.500570 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22585cd3-0041-4662-85fc-bf07a71e4d80 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:02.504494 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2047622-59f8-4964-abb4-24d5e0260e56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:02.505102 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2047622-59f8-4964-abb4-24d5e0260e56 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:02.505383 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2047622-59f8-4964-abb4-24d5e0260e56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:02.505661 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2047622-59f8-4964-abb4-24d5e0260e56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:02.515162 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2047622-59f8-4964-abb4-24d5e0260e56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:02.516722 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2047622-59f8-4964-abb4-24d5e0260e56 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:02.526523 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1373/2742] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:17:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:17:02.894036 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7cfe4-8d08-4c85-a86c-ee89df16abbb None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:02.895458 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2cd7cfe4-8d08-4c85-a86c-ee89df16abbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:17:02.907841 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7cfe4-8d08-4c85-a86c-ee89df16abbb None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:02.912435 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b7cee-f971-4d68-976e-65d6a5c3e86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:02.913098 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b7cee-f971-4d68-976e-65d6a5c3e86c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:02.913379 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b7cee-f971-4d68-976e-65d6a5c3e86c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:02.913654 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b7cee-f971-4d68-976e-65d6a5c3e86c admin admin] PATH_INFO: `/v3/projects/28c37a44130b4581841a5d0d7e52025d` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:02.915741 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b7cee-f971-4d68-976e-65d6a5c3e86c admin admin] RBAC: Authorizing `identity:delete_project(project_id=28c37a44130b4581841a5d0d7e52025d)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:02.917697 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b7cee-f971-4d68-976e-65d6a5c3e86c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:02.934753 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:17:03.046284 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-dd1b7cee-f971-4d68-976e-65d6a5c3e86c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28c37a44130b4581841a5d0d7e52025d', 'request_id': 'req-dd1b7cee-f971-4d68-976e-65d6a5c3e86c'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:03.119974 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-dd1b7cee-f971-4d68-976e-65d6a5c3e86c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28c37a44130b4581841a5d0d7e52025d', 'request_id': 'req-dd1b7cee-f971-4d68-976e-65d6a5c3e86c'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:03.125683 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-dd1b7cee-f971-4d68-976e-65d6a5c3e86c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28c37a44130b4581841a5d0d7e52025d', 'request_id': 'req-dd1b7cee-f971-4d68-976e-65d6a5c3e86c'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:03.137464 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-dd1b7cee-f971-4d68-976e-65d6a5c3e86c admin admin] The token cache is being invalidate because project 28c37a44130b4581841a5d0d7e52025d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:17:03.137771 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-dd1b7cee-f971-4d68-976e-65d6a5c3e86c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:03.138376 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-dd1b7cee-f971-4d68-976e-65d6a5c3e86c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:03.139851 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1370/2743] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:17:02 2022] DELETE /identity/v3/projects/28c37a44130b4581841a5d0d7e52025d => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:17:03.143848 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2047622-59f8-4964-abb4-24d5e0260e56 service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:03.154304 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2047622-59f8-4964-abb4-24d5e0260e56 service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:03.209025 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3de32-ab42-4b95-86fd-dea5c342e644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:03.209641 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3de32-ab42-4b95-86fd-dea5c342e644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:03.209920 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3de32-ab42-4b95-86fd-dea5c342e644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:03.210188 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3de32-ab42-4b95-86fd-dea5c342e644 admin admin] PATH_INFO: `/v3/users/d0850f597f784be99cdae731c3d0e5cd` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:03.212131 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e3de32-ab42-4b95-86fd-dea5c342e644 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0850f597f784be99cdae731c3d0e5cd)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:03.214064 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e3de32-ab42-4b95-86fd-dea5c342e644 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:03.305745 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:17:03.436862 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-91e3de32-ab42-4b95-86fd-dea5c342e644 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0850f597f784be99cdae731c3d0e5cd'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:03.437517 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-91e3de32-ab42-4b95-86fd-dea5c342e644 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0850f597f784be99cdae731c3d0e5cd'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:03.544171 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-91e3de32-ab42-4b95-86fd-dea5c342e644 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0850f597f784be99cdae731c3d0e5cd'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:03.544792 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-91e3de32-ab42-4b95-86fd-dea5c342e644 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0850f597f784be99cdae731c3d0e5cd'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:03.558072 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-91e3de32-ab42-4b95-86fd-dea5c342e644 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0850f597f784be99cdae731c3d0e5cd'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:03.571367 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1374/2744] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:17:03 2022] DELETE /identity/v3/users/d0850f597f784be99cdae731c3d0e5cd => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:17:03.939834 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b7cee-f971-4d68-976e-65d6a5c3e86c admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:03.941863 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-dd1b7cee-f971-4d68-976e-65d6a5c3e86c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:17:03.955215 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b7cee-f971-4d68-976e-65d6a5c3e86c admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:04.013326 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a2bb3-cc2e-4c3e-a231-dd329ab3ce52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:04.013950 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a2bb3-cc2e-4c3e-a231-dd329ab3ce52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:04.014314 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a2bb3-cc2e-4c3e-a231-dd329ab3ce52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:04.014596 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a2bb3-cc2e-4c3e-a231-dd329ab3ce52 admin admin] PATH_INFO: `/v3/projects/0d5becbfbc444d9483489251e59ce15c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:04.015680 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a2bb3-cc2e-4c3e-a231-dd329ab3ce52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d5becbfbc444d9483489251e59ce15c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:04.017942 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a2bb3-cc2e-4c3e-a231-dd329ab3ce52 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:04.133076 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1b8a2bb3-cc2e-4c3e-a231-dd329ab3ce52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d5becbfbc444d9483489251e59ce15c', 'request_id': 'req-1b8a2bb3-cc2e-4c3e-a231-dd329ab3ce52'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:04.210941 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1b8a2bb3-cc2e-4c3e-a231-dd329ab3ce52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d5becbfbc444d9483489251e59ce15c', 'request_id': 'req-1b8a2bb3-cc2e-4c3e-a231-dd329ab3ce52'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:04.217230 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1b8a2bb3-cc2e-4c3e-a231-dd329ab3ce52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d5becbfbc444d9483489251e59ce15c', 'request_id': 'req-1b8a2bb3-cc2e-4c3e-a231-dd329ab3ce52'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:04.230404 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1b8a2bb3-cc2e-4c3e-a231-dd329ab3ce52 admin admin] The token cache is being invalidate because project 0d5becbfbc444d9483489251e59ce15c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:17:04.230692 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1b8a2bb3-cc2e-4c3e-a231-dd329ab3ce52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:04.231397 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1b8a2bb3-cc2e-4c3e-a231-dd329ab3ce52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:04.232921 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1371/2745] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:17:03 2022] DELETE /identity/v3/projects/0d5becbfbc444d9483489251e59ce15c => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:17:04.236878 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3de32-ab42-4b95-86fd-dea5c342e644 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:04.238339 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-91e3de32-ab42-4b95-86fd-dea5c342e644 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:17:04.250895 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3de32-ab42-4b95-86fd-dea5c342e644 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:04.306951 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1995662-4661-474c-b0ef-b5e6a3fbe6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:04.307573 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1995662-4661-474c-b0ef-b5e6a3fbe6ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:04.307855 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1995662-4661-474c-b0ef-b5e6a3fbe6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:04.308133 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1995662-4661-474c-b0ef-b5e6a3fbe6ce admin admin] PATH_INFO: `/v3/users/ea1f3f870c4944f3ae2cc38f00f8a116` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:04.309281 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1995662-4661-474c-b0ef-b5e6a3fbe6ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea1f3f870c4944f3ae2cc38f00f8a116)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:04.311186 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1995662-4661-474c-b0ef-b5e6a3fbe6ce admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:04.482274 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b1995662-4661-474c-b0ef-b5e6a3fbe6ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea1f3f870c4944f3ae2cc38f00f8a116'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:04.482841 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b1995662-4661-474c-b0ef-b5e6a3fbe6ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea1f3f870c4944f3ae2cc38f00f8a116'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:04.559595 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b1995662-4661-474c-b0ef-b5e6a3fbe6ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea1f3f870c4944f3ae2cc38f00f8a116'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:04.560204 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b1995662-4661-474c-b0ef-b5e6a3fbe6ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea1f3f870c4944f3ae2cc38f00f8a116'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:04.573382 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-b1995662-4661-474c-b0ef-b5e6a3fbe6ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea1f3f870c4944f3ae2cc38f00f8a116'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:04.585522 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1375/2746] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:17:04 2022] DELETE /identity/v3/users/ea1f3f870c4944f3ae2cc38f00f8a116 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:17:04.934661 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a2bb3-cc2e-4c3e-a231-dd329ab3ce52 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:04.936089 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1b8a2bb3-cc2e-4c3e-a231-dd329ab3ce52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:17:04.951000 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a2bb3-cc2e-4c3e-a231-dd329ab3ce52 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:04.989845 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:17:05.018695 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35abd807-4863-4800-97fa-712cce77fa74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:05.019324 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abd807-4863-4800-97fa-712cce77fa74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:05.019609 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abd807-4863-4800-97fa-712cce77fa74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:05.019883 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abd807-4863-4800-97fa-712cce77fa74 admin admin] PATH_INFO: `/v3/projects/bb2f3de345d2454d838eb5ab953b6604` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:05.021062 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35abd807-4863-4800-97fa-712cce77fa74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb2f3de345d2454d838eb5ab953b6604)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:05.022955 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35abd807-4863-4800-97fa-712cce77fa74 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:05.126193 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-35abd807-4863-4800-97fa-712cce77fa74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb2f3de345d2454d838eb5ab953b6604', 'request_id': 'req-35abd807-4863-4800-97fa-712cce77fa74'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:05.193975 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-35abd807-4863-4800-97fa-712cce77fa74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb2f3de345d2454d838eb5ab953b6604', 'request_id': 'req-35abd807-4863-4800-97fa-712cce77fa74'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:05.199748 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-35abd807-4863-4800-97fa-712cce77fa74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb2f3de345d2454d838eb5ab953b6604', 'request_id': 'req-35abd807-4863-4800-97fa-712cce77fa74'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:05.211332 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-35abd807-4863-4800-97fa-712cce77fa74 admin admin] The token cache is being invalidate because project bb2f3de345d2454d838eb5ab953b6604 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:17:05.212225 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-35abd807-4863-4800-97fa-712cce77fa74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:05.212930 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-35abd807-4863-4800-97fa-712cce77fa74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:05.214406 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1372/2747] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:17:04 2022] DELETE /identity/v3/projects/bb2f3de345d2454d838eb5ab953b6604 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:17:05.219493 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c3ac7-97df-4b4f-b2a4-afb2462c23e9 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:05.219786 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c3ac7-97df-4b4f-b2a4-afb2462c23e9 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:05.220071 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c3ac7-97df-4b4f-b2a4-afb2462c23e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:05.241813 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-416c3ac7-97df-4b4f-b2a4-afb2462c23e9 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:17:05.243118 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-416c3ac7-97df-4b4f-b2a4-afb2462c23e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:17:05.262051 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:17:05.302010 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1376/2748] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:17:05.309988 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54abd4-d7af-480e-bea4-66708c579089 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:05.310280 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54abd4-d7af-480e-bea4-66708c579089 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:05.310558 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54abd4-d7af-480e-bea4-66708c579089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:05.330851 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-9c54abd4-d7af-480e-bea4-66708c579089 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:17:05.332063 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9c54abd4-d7af-480e-bea4-66708c579089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:17:05.340714 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1373/2749] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:17:05.344902 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c3ac7-97df-4b4f-b2a4-afb2462c23e9 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:05.354357 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c3ac7-97df-4b4f-b2a4-afb2462c23e9 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:05.359246 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30b479-5646-48b7-8740-e1912aeaab52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:05.359836 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30b479-5646-48b7-8740-e1912aeaab52 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:05.360110 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30b479-5646-48b7-8740-e1912aeaab52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:05.360376 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30b479-5646-48b7-8740-e1912aeaab52 admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:05.361049 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30b479-5646-48b7-8740-e1912aeaab52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:17:05.361344 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30b479-5646-48b7-8740-e1912aeaab52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:05.362923 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30b479-5646-48b7-8740-e1912aeaab52 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:05.375003 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1377/2750] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:17:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:17:05.394621 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54abd4-d7af-480e-bea4-66708c579089 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:05.407247 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54abd4-d7af-480e-bea4-66708c579089 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:05.412786 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66697f2-1ec2-43f4-b17e-fc67f1501707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:05.413408 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66697f2-1ec2-43f4-b17e-fc67f1501707 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:05.413695 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66697f2-1ec2-43f4-b17e-fc67f1501707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:05.413965 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66697f2-1ec2-43f4-b17e-fc67f1501707 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:05.414656 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66697f2-1ec2-43f4-b17e-fc67f1501707 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:05.416784 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66697f2-1ec2-43f4-b17e-fc67f1501707 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:05.575568 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1374/2751] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:17:05 2022] POST /identity/v3/projects => generated 385 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:17:05.579705 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30b479-5646-48b7-8740-e1912aeaab52 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:05.590319 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30b479-5646-48b7-8740-e1912aeaab52 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:05.646285 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a7890-cdc0-4a87-bd7a-bd8ba2b1ab4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:05.646890 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a7890-cdc0-4a87-bd7a-bd8ba2b1ab4f admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:05.647190 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a7890-cdc0-4a87-bd7a-bd8ba2b1ab4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:05.647447 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a7890-cdc0-4a87-bd7a-bd8ba2b1ab4f admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:05.648097 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a7890-cdc0-4a87-bd7a-bd8ba2b1ab4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:05.651177 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a7890-cdc0-4a87-bd7a-bd8ba2b1ab4f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:05.760847 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1378/2752] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:17:05 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:17:05.765038 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66697f2-1ec2-43f4-b17e-fc67f1501707 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:05.776411 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66697f2-1ec2-43f4-b17e-fc67f1501707 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:05.781020 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e2328b-eb2d-404a-a8af-3c28c1a56bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:05.781714 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2328b-eb2d-404a-a8af-3c28c1a56bc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:05.781996 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2328b-eb2d-404a-a8af-3c28c1a56bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:05.782557 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2328b-eb2d-404a-a8af-3c28c1a56bc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:05.783197 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e2328b-eb2d-404a-a8af-3c28c1a56bc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:05.784821 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e2328b-eb2d-404a-a8af-3c28c1a56bc2 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:05.795616 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1375/2753] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:17:05.799593 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a7890-cdc0-4a87-bd7a-bd8ba2b1ab4f admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:05.810459 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a7890-cdc0-4a87-bd7a-bd8ba2b1ab4f admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:05.815067 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15efd0-4e25-4313-8615-281da309b46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:05.820888 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15efd0-4e25-4313-8615-281da309b46b admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:05.820888 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15efd0-4e25-4313-8615-281da309b46b admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:05.820888 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15efd0-4e25-4313-8615-281da309b46b admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:05.820888 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15efd0-4e25-4313-8615-281da309b46b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:05.820888 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15efd0-4e25-4313-8615-281da309b46b admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:05.829667 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1379/2754] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:17:05.833916 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e2328b-eb2d-404a-a8af-3c28c1a56bc2 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:05.844645 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e2328b-eb2d-404a-a8af-3c28c1a56bc2 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:05.850349 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd262ff-32a5-40f8-9159-91fd2662c221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:05.850349 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd262ff-32a5-40f8-9159-91fd2662c221 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:05.850349 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd262ff-32a5-40f8-9159-91fd2662c221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:05.850658 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd262ff-32a5-40f8-9159-91fd2662c221 admin admin] PATH_INFO: `/v3/projects/47b09ae96cff43a4874a43ce22d3c2ca/users/6b4fa8afdbda4a47b6f89fc81f320f39/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:05.865590 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd262ff-32a5-40f8-9159-91fd2662c221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47b09ae96cff43a4874a43ce22d3c2ca, user_id=6b4fa8afdbda4a47b6f89fc81f320f39, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:05.869466 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd262ff-32a5-40f8-9159-91fd2662c221 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:05.932739 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1376/2755] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:17:05 2022] PUT /identity/v3/projects/47b09ae96cff43a4874a43ce22d3c2ca/users/6b4fa8afdbda4a47b6f89fc81f320f39/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:17:05.943122 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189baa1-df05-410b-94ab-7811e96f348a None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:05.946713 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189baa1-df05-410b-94ab-7811e96f348a None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:05.946713 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189baa1-df05-410b-94ab-7811e96f348a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:05.971430 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-7189baa1-df05-410b-94ab-7811e96f348a None None] MFA Rules not processed for user `6b4fa8afdbda4a47b6f89fc81f320f39`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:17:05.975371 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-7189baa1-df05-410b-94ab-7811e96f348a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:17:06.035040 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1380/2756] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:17:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:17:06.046601 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd262ff-32a5-40f8-9159-91fd2662c221 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:06.048324 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-cdd262ff-32a5-40f8-9159-91fd2662c221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:17:06.108194 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd262ff-32a5-40f8-9159-91fd2662c221 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:06.111816 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fda72d-9b13-46fd-81dd-63d01cd2a97e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:06.112418 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fda72d-9b13-46fd-81dd-63d01cd2a97e service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:06.112730 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fda72d-9b13-46fd-81dd-63d01cd2a97e service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:06.113002 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fda72d-9b13-46fd-81dd-63d01cd2a97e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:06.122070 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fda72d-9b13-46fd-81dd-63d01cd2a97e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:06.123482 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fda72d-9b13-46fd-81dd-63d01cd2a97e service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:06.132153 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1377/2757] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:17:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:17:09.183400 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7189baa1-df05-410b-94ab-7811e96f348a None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:09.204087 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:17:09.246224 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7189baa1-df05-410b-94ab-7811e96f348a None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:09.250778 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce71962a-0f55-47e7-8217-b1c63a1c46db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:09.251565 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71962a-0f55-47e7-8217-b1c63a1c46db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:09.251674 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71962a-0f55-47e7-8217-b1c63a1c46db admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:09.251936 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71962a-0f55-47e7-8217-b1c63a1c46db admin admin] PATH_INFO: `/v3/users/6b4fa8afdbda4a47b6f89fc81f320f39` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:09.253066 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce71962a-0f55-47e7-8217-b1c63a1c46db admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b4fa8afdbda4a47b6f89fc81f320f39)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:09.254975 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce71962a-0f55-47e7-8217-b1c63a1c46db admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:09.467095 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ce71962a-0f55-47e7-8217-b1c63a1c46db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b4fa8afdbda4a47b6f89fc81f320f39'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:09.467694 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ce71962a-0f55-47e7-8217-b1c63a1c46db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b4fa8afdbda4a47b6f89fc81f320f39'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:09.538047 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ce71962a-0f55-47e7-8217-b1c63a1c46db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b4fa8afdbda4a47b6f89fc81f320f39'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:09.538671 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ce71962a-0f55-47e7-8217-b1c63a1c46db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b4fa8afdbda4a47b6f89fc81f320f39'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:09.551894 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ce71962a-0f55-47e7-8217-b1c63a1c46db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b4fa8afdbda4a47b6f89fc81f320f39'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:09.564332 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1381/2758] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:17:09 2022] DELETE /identity/v3/users/6b4fa8afdbda4a47b6f89fc81f320f39 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:17:09.569059 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31125151-aac5-4342-b263-f4d659ab3ba4 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:09.569679 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31125151-aac5-4342-b263-f4d659ab3ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:09.569679 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31125151-aac5-4342-b263-f4d659ab3ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:09.576713 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:17:09.589241 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-31125151-aac5-4342-b263-f4d659ab3ba4 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:17:09.590433 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-31125151-aac5-4342-b263-f4d659ab3ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:17:09.648202 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1378/2759] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:17:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:17:09.658516 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce71962a-0f55-47e7-8217-b1c63a1c46db admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:09.659922 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ce71962a-0f55-47e7-8217-b1c63a1c46db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:17:09.717728 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce71962a-0f55-47e7-8217-b1c63a1c46db admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:09.721167 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b99064-4ecc-4800-a0fc-57ebb6885453 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:09.721771 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b99064-4ecc-4800-a0fc-57ebb6885453 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:09.722041 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b99064-4ecc-4800-a0fc-57ebb6885453 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:09.722320 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b99064-4ecc-4800-a0fc-57ebb6885453 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:09.731916 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b99064-4ecc-4800-a0fc-57ebb6885453 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:09.733383 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b99064-4ecc-4800-a0fc-57ebb6885453 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:09.742598 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1382/2760] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:17:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:17:10.197830 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31125151-aac5-4342-b263-f4d659ab3ba4 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:10.199265 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-31125151-aac5-4342-b263-f4d659ab3ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:17:10.211570 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31125151-aac5-4342-b263-f4d659ab3ba4 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:10.216759 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3f88f5-a0db-4dd8-9ed1-d37ddb1241c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:10.217407 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f88f5-a0db-4dd8-9ed1-d37ddb1241c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:10.217694 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f88f5-a0db-4dd8-9ed1-d37ddb1241c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:10.217966 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f88f5-a0db-4dd8-9ed1-d37ddb1241c3 admin admin] PATH_INFO: `/v3/projects/47b09ae96cff43a4874a43ce22d3c2ca` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:10.223623 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3f88f5-a0db-4dd8-9ed1-d37ddb1241c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47b09ae96cff43a4874a43ce22d3c2ca)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:10.223623 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3f88f5-a0db-4dd8-9ed1-d37ddb1241c3 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:10.333214 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8a3f88f5-a0db-4dd8-9ed1-d37ddb1241c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47b09ae96cff43a4874a43ce22d3c2ca', 'request_id': 'req-8a3f88f5-a0db-4dd8-9ed1-d37ddb1241c3'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:10.412871 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8a3f88f5-a0db-4dd8-9ed1-d37ddb1241c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47b09ae96cff43a4874a43ce22d3c2ca', 'request_id': 'req-8a3f88f5-a0db-4dd8-9ed1-d37ddb1241c3'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:10.419059 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8a3f88f5-a0db-4dd8-9ed1-d37ddb1241c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47b09ae96cff43a4874a43ce22d3c2ca', 'request_id': 'req-8a3f88f5-a0db-4dd8-9ed1-d37ddb1241c3'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:10.430493 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8a3f88f5-a0db-4dd8-9ed1-d37ddb1241c3 admin admin] The token cache is being invalidate because project 47b09ae96cff43a4874a43ce22d3c2ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:17:10.430786 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8a3f88f5-a0db-4dd8-9ed1-d37ddb1241c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:10.432028 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8a3f88f5-a0db-4dd8-9ed1-d37ddb1241c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:10.433775 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1379/2761] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:17:10 2022] DELETE /identity/v3/projects/47b09ae96cff43a4874a43ce22d3c2ca => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:17:10.438717 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e8c19-4d93-46d8-ae36-b23385533204 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:10.439186 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e8c19-4d93-46d8-ae36-b23385533204 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:10.439284 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e8c19-4d93-46d8-ae36-b23385533204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:10.458960 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-b48e8c19-4d93-46d8-ae36-b23385533204 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:17:10.460152 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b48e8c19-4d93-46d8-ae36-b23385533204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:17:10.498507 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:17:10.520178 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1383/2762] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:17:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:17:10.529272 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ae7c2-a5c0-4727-976a-1ee02c317259 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:10.529569 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ae7c2-a5c0-4727-976a-1ee02c317259 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:10.536112 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ae7c2-a5c0-4727-976a-1ee02c317259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:10.550239 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-cb4ae7c2-a5c0-4727-976a-1ee02c317259 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:17:10.551451 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-cb4ae7c2-a5c0-4727-976a-1ee02c317259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:17:10.560409 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1380/2763] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:17:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:17:10.565583 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e8c19-4d93-46d8-ae36-b23385533204 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:10.575099 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e8c19-4d93-46d8-ae36-b23385533204 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:10.579870 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f77113-c603-4f5f-af04-5db7a12b933d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:10.580494 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f77113-c603-4f5f-af04-5db7a12b933d admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:10.580783 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f77113-c603-4f5f-af04-5db7a12b933d admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:10.581060 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f77113-c603-4f5f-af04-5db7a12b933d admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:10.581711 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f77113-c603-4f5f-af04-5db7a12b933d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:17:10.582018 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f77113-c603-4f5f-af04-5db7a12b933d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:10.583694 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f77113-c603-4f5f-af04-5db7a12b933d admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:10.595141 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1384/2764] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:17:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 04 01:17:10.599498 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ae7c2-a5c0-4727-976a-1ee02c317259 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:10.609313 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ae7c2-a5c0-4727-976a-1ee02c317259 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:10.614032 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479005de-0729-4b2a-b602-338f2d7082d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:10.614625 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479005de-0729-4b2a-b602-338f2d7082d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:10.614909 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479005de-0729-4b2a-b602-338f2d7082d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:10.615185 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479005de-0729-4b2a-b602-338f2d7082d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:10.615814 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479005de-0729-4b2a-b602-338f2d7082d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:10.617742 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479005de-0729-4b2a-b602-338f2d7082d1 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:10.728797 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1381/2765] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:17:10 2022] POST /identity/v3/projects => generated 389 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:17:10.733507 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f77113-c603-4f5f-af04-5db7a12b933d admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:10.745526 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f77113-c603-4f5f-af04-5db7a12b933d admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:10.802582 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9f9d4-8c2f-49f1-ad89-acd51a80d24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:10.803118 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9f9d4-8c2f-49f1-ad89-acd51a80d24b admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:10.803404 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9f9d4-8c2f-49f1-ad89-acd51a80d24b admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:10.803683 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9f9d4-8c2f-49f1-ad89-acd51a80d24b admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:10.804309 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a9f9d4-8c2f-49f1-ad89-acd51a80d24b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:10.806427 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a9f9d4-8c2f-49f1-ad89-acd51a80d24b admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:10.927090 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1385/2766] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:17:10 2022] POST /identity/v3/users => generated 295 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:17:10.931429 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479005de-0729-4b2a-b602-338f2d7082d1 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:10.941653 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479005de-0729-4b2a-b602-338f2d7082d1 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:10.946239 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971f0d1d-f2ab-4204-a88c-0d5a0dca2b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:10.946839 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f0d1d-f2ab-4204-a88c-0d5a0dca2b5f admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:10.947203 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f0d1d-f2ab-4204-a88c-0d5a0dca2b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:10.947631 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f0d1d-f2ab-4204-a88c-0d5a0dca2b5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:10.948353 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971f0d1d-f2ab-4204-a88c-0d5a0dca2b5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:10.950000 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971f0d1d-f2ab-4204-a88c-0d5a0dca2b5f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:10.959651 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1382/2767] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:17:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:17:10.963990 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9f9d4-8c2f-49f1-ad89-acd51a80d24b admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:10.973829 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9f9d4-8c2f-49f1-ad89-acd51a80d24b admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:10.979153 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3112e4c7-50d3-4b5b-97ab-600e9e0cee36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:10.979740 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3112e4c7-50d3-4b5b-97ab-600e9e0cee36 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:10.980016 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3112e4c7-50d3-4b5b-97ab-600e9e0cee36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:10.980289 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3112e4c7-50d3-4b5b-97ab-600e9e0cee36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:10.980951 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3112e4c7-50d3-4b5b-97ab-600e9e0cee36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:10.982523 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3112e4c7-50d3-4b5b-97ab-600e9e0cee36 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:10.993982 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1386/2768] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:17:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:17:10.998067 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971f0d1d-f2ab-4204-a88c-0d5a0dca2b5f admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:11.008022 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971f0d1d-f2ab-4204-a88c-0d5a0dca2b5f admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:11.013299 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350894df-9d1d-4ea3-b7c0-00b5d75b3a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:11.013941 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350894df-9d1d-4ea3-b7c0-00b5d75b3a58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:11.014224 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350894df-9d1d-4ea3-b7c0-00b5d75b3a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:11.014501 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350894df-9d1d-4ea3-b7c0-00b5d75b3a58 admin admin] PATH_INFO: `/v3/projects/ccb026ae713e45a0b0e4215fe0f09ee0/users/b7fbbf23102c4fcb8dd53dfa3f294402/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:11.029804 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350894df-9d1d-4ea3-b7c0-00b5d75b3a58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccb026ae713e45a0b0e4215fe0f09ee0, user_id=b7fbbf23102c4fcb8dd53dfa3f294402, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:11.032513 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350894df-9d1d-4ea3-b7c0-00b5d75b3a58 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:11.126772 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1383/2769] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:17:10 2022] PUT /identity/v3/projects/ccb026ae713e45a0b0e4215fe0f09ee0/users/b7fbbf23102c4fcb8dd53dfa3f294402/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:17:11.137367 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc345898-2f52-4766-a47d-c51b81d21339 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:11.143295 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc345898-2f52-4766-a47d-c51b81d21339 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:11.143295 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc345898-2f52-4766-a47d-c51b81d21339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:11.165372 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-bc345898-2f52-4766-a47d-c51b81d21339 None None] MFA Rules not processed for user `b7fbbf23102c4fcb8dd53dfa3f294402`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:17:11.167275 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-bc345898-2f52-4766-a47d-c51b81d21339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:17:11.229952 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1387/2770] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:17:11 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:17:11.240699 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350894df-9d1d-4ea3-b7c0-00b5d75b3a58 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:11.242402 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-350894df-9d1d-4ea3-b7c0-00b5d75b3a58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:17:11.268367 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:17:11.298987 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350894df-9d1d-4ea3-b7c0-00b5d75b3a58 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:11.302930 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf8866-174f-48ff-97da-25a4da1b3243 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:11.303449 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf8866-174f-48ff-97da-25a4da1b3243 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:11.303732 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf8866-174f-48ff-97da-25a4da1b3243 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:11.304009 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf8866-174f-48ff-97da-25a4da1b3243 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:11.312936 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cf8866-174f-48ff-97da-25a4da1b3243 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:11.314354 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cf8866-174f-48ff-97da-25a4da1b3243 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:11.323596 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1384/2771] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:17:11 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:17:21.227155 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=80,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:17:21.360863 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=16 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:17:22.749673 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc345898-2f52-4766-a47d-c51b81d21339 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:22.811586 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc345898-2f52-4766-a47d-c51b81d21339 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:22.816150 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed302eb7-d5e3-4099-af3d-ca192b634c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:22.816804 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed302eb7-d5e3-4099-af3d-ca192b634c95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:22.817085 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed302eb7-d5e3-4099-af3d-ca192b634c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:22.817358 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed302eb7-d5e3-4099-af3d-ca192b634c95 admin admin] PATH_INFO: `/v3/users/b7fbbf23102c4fcb8dd53dfa3f294402` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:22.818469 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed302eb7-d5e3-4099-af3d-ca192b634c95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7fbbf23102c4fcb8dd53dfa3f294402)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:22.820406 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed302eb7-d5e3-4099-af3d-ca192b634c95 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:22.972652 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ed302eb7-d5e3-4099-af3d-ca192b634c95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7fbbf23102c4fcb8dd53dfa3f294402'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:22.973294 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ed302eb7-d5e3-4099-af3d-ca192b634c95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7fbbf23102c4fcb8dd53dfa3f294402'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:23.039593 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ed302eb7-d5e3-4099-af3d-ca192b634c95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7fbbf23102c4fcb8dd53dfa3f294402'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:23.040230 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ed302eb7-d5e3-4099-af3d-ca192b634c95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7fbbf23102c4fcb8dd53dfa3f294402'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:23.053468 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-ed302eb7-d5e3-4099-af3d-ca192b634c95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7fbbf23102c4fcb8dd53dfa3f294402'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:23.065717 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1388/2772] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:17:22 2022] DELETE /identity/v3/users/b7fbbf23102c4fcb8dd53dfa3f294402 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:17:23.070331 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c52da-027b-4286-9001-15f34f38664a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:23.070837 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c52da-027b-4286-9001-15f34f38664a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:23.070939 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c52da-027b-4286-9001-15f34f38664a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:23.090367 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-e34c52da-027b-4286-9001-15f34f38664a None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:17:23.091549 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e34c52da-027b-4286-9001-15f34f38664a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:17:23.148682 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1385/2773] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:17:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:17:23.159063 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed302eb7-d5e3-4099-af3d-ca192b634c95 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:23.160482 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ed302eb7-d5e3-4099-af3d-ca192b634c95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:17:23.216992 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed302eb7-d5e3-4099-af3d-ca192b634c95 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:23.220628 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bfb846-dcd3-4dff-917f-566999f184f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:23.221233 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfb846-dcd3-4dff-917f-566999f184f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:23.221508 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfb846-dcd3-4dff-917f-566999f184f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:23.221776 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfb846-dcd3-4dff-917f-566999f184f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:23.232619 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bfb846-dcd3-4dff-917f-566999f184f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:23.234078 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bfb846-dcd3-4dff-917f-566999f184f0 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:23.243888 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1389/2774] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:17:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:17:23.643377 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c52da-027b-4286-9001-15f34f38664a None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:23.645642 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e34c52da-027b-4286-9001-15f34f38664a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:17:23.656477 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c52da-027b-4286-9001-15f34f38664a None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:23.661286 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e150a11-f0ef-4d8c-9223-3330773b0f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:23.661912 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e150a11-f0ef-4d8c-9223-3330773b0f91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:23.662195 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e150a11-f0ef-4d8c-9223-3330773b0f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:23.662463 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e150a11-f0ef-4d8c-9223-3330773b0f91 admin admin] PATH_INFO: `/v3/projects/ccb026ae713e45a0b0e4215fe0f09ee0` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:23.663574 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e150a11-f0ef-4d8c-9223-3330773b0f91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccb026ae713e45a0b0e4215fe0f09ee0)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:23.665644 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e150a11-f0ef-4d8c-9223-3330773b0f91 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:23.783224 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2e150a11-f0ef-4d8c-9223-3330773b0f91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccb026ae713e45a0b0e4215fe0f09ee0', 'request_id': 'req-2e150a11-f0ef-4d8c-9223-3330773b0f91'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:23.840251 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2e150a11-f0ef-4d8c-9223-3330773b0f91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccb026ae713e45a0b0e4215fe0f09ee0', 'request_id': 'req-2e150a11-f0ef-4d8c-9223-3330773b0f91'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:23.846206 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2e150a11-f0ef-4d8c-9223-3330773b0f91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccb026ae713e45a0b0e4215fe0f09ee0', 'request_id': 'req-2e150a11-f0ef-4d8c-9223-3330773b0f91'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:23.857479 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2e150a11-f0ef-4d8c-9223-3330773b0f91 admin admin] The token cache is being invalidate because project ccb026ae713e45a0b0e4215fe0f09ee0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:17:23.857771 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2e150a11-f0ef-4d8c-9223-3330773b0f91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:23.859843 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2e150a11-f0ef-4d8c-9223-3330773b0f91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:17:23.859843 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1386/2775] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:17:23 2022] DELETE /identity/v3/projects/ccb026ae713e45a0b0e4215fe0f09ee0 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:17:23.864753 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c2e5a-eaa6-4299-9925-54f6d313fa76 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:23.865212 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c2e5a-eaa6-4299-9925-54f6d313fa76 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:23.865310 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c2e5a-eaa6-4299-9925-54f6d313fa76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:23.885400 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-4e3c2e5a-eaa6-4299-9925-54f6d313fa76 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:17:23.887550 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-4e3c2e5a-eaa6-4299-9925-54f6d313fa76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:17:23.904216 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:17:23.944439 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1390/2776] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:17:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:17:23.954294 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e1db1-b0c9-4ae7-9225-1449c32b4adf None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:23.954581 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e1db1-b0c9-4ae7-9225-1449c32b4adf None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:23.954860 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e1db1-b0c9-4ae7-9225-1449c32b4adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:23.974440 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-b16e1db1-b0c9-4ae7-9225-1449c32b4adf None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:17:23.975644 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b16e1db1-b0c9-4ae7-9225-1449c32b4adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:17:23.984355 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1387/2777] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:17:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:17:24.044289 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3c2e5a-eaa6-4299-9925-54f6d313fa76 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:24.054069 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3c2e5a-eaa6-4299-9925-54f6d313fa76 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:24.059306 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8da704-c1f0-4bb7-89fe-2b6b49e9c79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:24.059900 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8da704-c1f0-4bb7-89fe-2b6b49e9c79e admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:24.060177 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8da704-c1f0-4bb7-89fe-2b6b49e9c79e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:24.060449 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8da704-c1f0-4bb7-89fe-2b6b49e9c79e admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:24.061120 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8da704-c1f0-4bb7-89fe-2b6b49e9c79e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:17:24.061421 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8da704-c1f0-4bb7-89fe-2b6b49e9c79e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:24.062999 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8da704-c1f0-4bb7-89fe-2b6b49e9c79e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:24.074279 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1391/2778] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:17:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:17:24.078560 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16e1db1-b0c9-4ae7-9225-1449c32b4adf None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:24.087619 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16e1db1-b0c9-4ae7-9225-1449c32b4adf None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:24.092339 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2413a1-f795-4baa-bd07-4bf5f1cc1247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:24.092960 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2413a1-f795-4baa-bd07-4bf5f1cc1247 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:24.093236 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2413a1-f795-4baa-bd07-4bf5f1cc1247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:24.093508 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2413a1-f795-4baa-bd07-4bf5f1cc1247 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:24.094143 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2413a1-f795-4baa-bd07-4bf5f1cc1247 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:24.096026 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2413a1-f795-4baa-bd07-4bf5f1cc1247 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:24.195346 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1388/2779] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:17:24 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:17:24.199793 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8da704-c1f0-4bb7-89fe-2b6b49e9c79e admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:24.209750 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8da704-c1f0-4bb7-89fe-2b6b49e9c79e admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:24.266514 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12d8612-334a-442c-bd0b-67ee4736d3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:24.267122 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d8612-334a-442c-bd0b-67ee4736d3d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:24.267404 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d8612-334a-442c-bd0b-67ee4736d3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:24.267677 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d8612-334a-442c-bd0b-67ee4736d3d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:24.268299 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12d8612-334a-442c-bd0b-67ee4736d3d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:24.270315 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12d8612-334a-442c-bd0b-67ee4736d3d1 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:24.373545 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1392/2780] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:17:24 2022] POST /identity/v3/users => generated 292 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:17:24.377926 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2413a1-f795-4baa-bd07-4bf5f1cc1247 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:24.389269 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2413a1-f795-4baa-bd07-4bf5f1cc1247 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:24.393940 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0667fa-334b-487f-9883-adc22aa368d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:24.394529 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0667fa-334b-487f-9883-adc22aa368d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:24.394810 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0667fa-334b-487f-9883-adc22aa368d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:24.395084 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0667fa-334b-487f-9883-adc22aa368d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:24.395718 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0667fa-334b-487f-9883-adc22aa368d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:24.397322 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0667fa-334b-487f-9883-adc22aa368d7 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:24.408297 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1389/2781] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:17:24.412286 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12d8612-334a-442c-bd0b-67ee4736d3d1 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:24.423087 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12d8612-334a-442c-bd0b-67ee4736d3d1 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:24.427672 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98875a1c-7112-4abc-a4c3-86e1c1704791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:24.428259 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98875a1c-7112-4abc-a4c3-86e1c1704791 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:24.428574 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98875a1c-7112-4abc-a4c3-86e1c1704791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:24.428854 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98875a1c-7112-4abc-a4c3-86e1c1704791 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:24.429491 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98875a1c-7112-4abc-a4c3-86e1c1704791 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:24.431076 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98875a1c-7112-4abc-a4c3-86e1c1704791 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:24.441447 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1393/2782] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:17:24.445466 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0667fa-334b-487f-9883-adc22aa368d7 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:24.455167 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0667fa-334b-487f-9883-adc22aa368d7 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:24.459946 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00451f3-e8d4-448b-996f-071868881c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:24.460629 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00451f3-e8d4-448b-996f-071868881c9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:24.460909 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00451f3-e8d4-448b-996f-071868881c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:24.461185 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00451f3-e8d4-448b-996f-071868881c9e admin admin] PATH_INFO: `/v3/projects/5a4534146f2f4d5f8f2ed8a2b9d1fb3e/users/35e7aa6333ae45d8af239fe22fc374f2/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:24.476410 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00451f3-e8d4-448b-996f-071868881c9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a4534146f2f4d5f8f2ed8a2b9d1fb3e, user_id=35e7aa6333ae45d8af239fe22fc374f2, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:24.479055 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00451f3-e8d4-448b-996f-071868881c9e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:24.582646 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1390/2783] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:17:24 2022] PUT /identity/v3/projects/5a4534146f2f4d5f8f2ed8a2b9d1fb3e/users/35e7aa6333ae45d8af239fe22fc374f2/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:17:24.592265 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd99e2c-e60a-4d28-9144-0bcbcfb4a051 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:24.592578 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd99e2c-e60a-4d28-9144-0bcbcfb4a051 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:24.592853 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd99e2c-e60a-4d28-9144-0bcbcfb4a051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:24.612101 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-afd99e2c-e60a-4d28-9144-0bcbcfb4a051 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:17:24.613329 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-afd99e2c-e60a-4d28-9144-0bcbcfb4a051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:17:24.672590 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1394/2784] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:17:24.682178 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00451f3-e8d4-448b-996f-071868881c9e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:24.683985 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e00451f3-e8d4-448b-996f-071868881c9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:17:24.705887 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:17:24.740604 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00451f3-e8d4-448b-996f-071868881c9e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:24.744690 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c87de1c-e523-4635-b1da-c3d6ccdc92f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:24.745304 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87de1c-e523-4635-b1da-c3d6ccdc92f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:24.745584 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87de1c-e523-4635-b1da-c3d6ccdc92f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:24.745855 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87de1c-e523-4635-b1da-c3d6ccdc92f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:24.754539 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c87de1c-e523-4635-b1da-c3d6ccdc92f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:24.757413 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c87de1c-e523-4635-b1da-c3d6ccdc92f8 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:24.765826 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1391/2785] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:17:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:17:29.946176 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a9b08-8370-456e-b3cd-db4d7ed211c5 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:29.946704 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a9b08-8370-456e-b3cd-db4d7ed211c5 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:29.946817 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a9b08-8370-456e-b3cd-db4d7ed211c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:29.982115 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-796a9b08-8370-456e-b3cd-db4d7ed211c5 None None] MFA Rules not processed for user `35e7aa6333ae45d8af239fe22fc374f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:17:29.983362 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-796a9b08-8370-456e-b3cd-db4d7ed211c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:17:30.004117 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:17:30.064367 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1395/2786] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:17:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:17:30.069075 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c87de1c-e523-4635-b1da-c3d6ccdc92f8 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:30.080352 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c87de1c-e523-4635-b1da-c3d6ccdc92f8 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:30.085457 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b832e-85a6-4253-85f2-a98959688658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:30.086247 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b832e-85a6-4253-85f2-a98959688658 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:30.086473 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b832e-85a6-4253-85f2-a98959688658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:30.086748 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b832e-85a6-4253-85f2-a98959688658 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:30.087403 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497b832e-85a6-4253-85f2-a98959688658 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:30.090072 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497b832e-85a6-4253-85f2-a98959688658 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:30.195606 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1392/2787] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:17:30 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:17:30.196796 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a9b08-8370-456e-b3cd-db4d7ed211c5 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:30.208052 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a9b08-8370-456e-b3cd-db4d7ed211c5 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:30.270528 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48621f-66e4-4309-be76-9b51c5d65404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:30.271139 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48621f-66e4-4309-be76-9b51c5d65404 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:30.271417 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48621f-66e4-4309-be76-9b51c5d65404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:30.271691 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48621f-66e4-4309-be76-9b51c5d65404 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:30.272356 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf48621f-66e4-4309-be76-9b51c5d65404 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:30.274310 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf48621f-66e4-4309-be76-9b51c5d65404 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:30.339311 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1396/2788] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:17:30 2022] POST /identity/v3/users => generated 299 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:17:30.343479 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b832e-85a6-4253-85f2-a98959688658 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:30.354384 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b832e-85a6-4253-85f2-a98959688658 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:30.359527 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede04ef-fc55-42fa-a006-08359a45b035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:30.360168 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede04ef-fc55-42fa-a006-08359a45b035 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:30.360611 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede04ef-fc55-42fa-a006-08359a45b035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:30.360889 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede04ef-fc55-42fa-a006-08359a45b035 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:30.361557 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede04ef-fc55-42fa-a006-08359a45b035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:30.363200 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede04ef-fc55-42fa-a006-08359a45b035 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:30.375676 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1393/2789] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:17:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:17:30.379927 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48621f-66e4-4309-be76-9b51c5d65404 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:30.390557 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48621f-66e4-4309-be76-9b51c5d65404 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:30.395562 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab5c10-641a-490b-b5e7-76d82307bfa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:30.396268 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab5c10-641a-490b-b5e7-76d82307bfa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:30.396585 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab5c10-641a-490b-b5e7-76d82307bfa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:30.396860 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab5c10-641a-490b-b5e7-76d82307bfa8 admin admin] PATH_INFO: `/v3/projects/a4819545ee3341588c2e82e386e89582/users/7a727c0cabe748c09c942c22abfbb7f2/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:30.412903 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab5c10-641a-490b-b5e7-76d82307bfa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4819545ee3341588c2e82e386e89582, user_id=7a727c0cabe748c09c942c22abfbb7f2, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:30.415522 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab5c10-641a-490b-b5e7-76d82307bfa8 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:30.475126 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1397/2790] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:17:30 2022] PUT /identity/v3/projects/a4819545ee3341588c2e82e386e89582/users/7a727c0cabe748c09c942c22abfbb7f2/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:17:30.479063 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede04ef-fc55-42fa-a006-08359a45b035 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:30.489872 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede04ef-fc55-42fa-a006-08359a45b035 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:30.546281 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a35b69-2ea7-4399-8b64-baf35919bb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:30.546909 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a35b69-2ea7-4399-8b64-baf35919bb71 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:30.547247 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a35b69-2ea7-4399-8b64-baf35919bb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:30.547522 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a35b69-2ea7-4399-8b64-baf35919bb71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:30.548158 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a35b69-2ea7-4399-8b64-baf35919bb71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:30.549794 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a35b69-2ea7-4399-8b64-baf35919bb71 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:30.559829 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1394/2791] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:17:30 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:17:30.564054 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab5c10-641a-490b-b5e7-76d82307bfa8 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:30.574334 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab5c10-641a-490b-b5e7-76d82307bfa8 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:30.578971 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155433e6-7c8a-4f9f-8aa6-455a4d341bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:30.579611 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155433e6-7c8a-4f9f-8aa6-455a4d341bb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:30.579891 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155433e6-7c8a-4f9f-8aa6-455a4d341bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:30.580166 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155433e6-7c8a-4f9f-8aa6-455a4d341bb0 admin admin] PATH_INFO: `/v3/projects/a4819545ee3341588c2e82e386e89582/users/7a727c0cabe748c09c942c22abfbb7f2/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:30.582123 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155433e6-7c8a-4f9f-8aa6-455a4d341bb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4819545ee3341588c2e82e386e89582, user_id=7a727c0cabe748c09c942c22abfbb7f2, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:30.584778 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155433e6-7c8a-4f9f-8aa6-455a4d341bb0 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:30.595971 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1398/2792] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:17:30 2022] PUT /identity/v3/projects/a4819545ee3341588c2e82e386e89582/users/7a727c0cabe748c09c942c22abfbb7f2/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:17:30.623227 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a35b69-2ea7-4399-8b64-baf35919bb71 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:30.679081 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a35b69-2ea7-4399-8b64-baf35919bb71 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:30.719938 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:17:30.728954 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3138a95-36d3-45db-997c-0f65e62c9367 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:30.730209 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3138a95-36d3-45db-997c-0f65e62c9367 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:30.730495 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3138a95-36d3-45db-997c-0f65e62c9367 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:30.730776 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3138a95-36d3-45db-997c-0f65e62c9367 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:30.732654 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b3138a95-36d3-45db-997c-0f65e62c9367 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:17:30.742233 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3138a95-36d3-45db-997c-0f65e62c9367 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:30.743660 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3138a95-36d3-45db-997c-0f65e62c9367 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:30.752175 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1395/2793] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:17:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:17:35.782708 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5cbb7c-e4cb-4ddc-82d5-7de8c5361b30 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:35.783204 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5cbb7c-e4cb-4ddc-82d5-7de8c5361b30 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:35.783204 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5cbb7c-e4cb-4ddc-82d5-7de8c5361b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:35.810904 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-3b5cbb7c-e4cb-4ddc-82d5-7de8c5361b30 None None] MFA Rules not processed for user `7a727c0cabe748c09c942c22abfbb7f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:17:35.812144 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-3b5cbb7c-e4cb-4ddc-82d5-7de8c5361b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:17:35.873560 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:17:35.882666 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1399/2794] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:17:35 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:17:35.899054 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3138a95-36d3-45db-997c-0f65e62c9367 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:35.900543 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b3138a95-36d3-45db-997c-0f65e62c9367 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:17:35.959578 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3138a95-36d3-45db-997c-0f65e62c9367 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:35.963154 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81eed0-869e-40d7-9ac0-d02c53f28cf2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1179503bdab547c99175a27b7ce27b9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:35.964374 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81eed0-869e-40d7-9ac0-d02c53f28cf2 service glance] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:35.964777 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81eed0-869e-40d7-9ac0-d02c53f28cf2 service glance] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:35.965053 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81eed0-869e-40d7-9ac0-d02c53f28cf2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:36.024622 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e81eed0-869e-40d7-9ac0-d02c53f28cf2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:36.026065 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e81eed0-869e-40d7-9ac0-d02c53f28cf2 service glance] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:36.036343 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1396/2795] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:17:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 01:17:36.523171 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef98e47-e5b7-4f53-b785-056abee39369 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:36.525629 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef98e47-e5b7-4f53-b785-056abee39369 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:36.525629 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef98e47-e5b7-4f53-b785-056abee39369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:36.581743 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-aef98e47-e5b7-4f53-b785-056abee39369 None None] MFA Rules not processed for user `d6d01ca673a1404c9b4cea51036c595c`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:17:36.582980 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-aef98e47-e5b7-4f53-b785-056abee39369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:17:36.653365 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1400/2796] 10.222.0.27 () {58 vars in 1009 bytes} [Sun Sep 4 01:17:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:17:36.804177 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81eed0-869e-40d7-9ac0-d02c53f28cf2 service glance] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:36.805629 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0e81eed0-869e-40d7-9ac0-d02c53f28cf2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:17:36.931673 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81eed0-869e-40d7-9ac0-d02c53f28cf2 service glance] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:36.937407 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d04fd-6836-41af-9510-766f8f33fa3f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2297d8b3fe8e47eb9fd39a6e3eea8f54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:36.938015 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d04fd-6836-41af-9510-766f8f33fa3f service swift] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:36.938295 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d04fd-6836-41af-9510-766f8f33fa3f service swift] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:36.938569 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d04fd-6836-41af-9510-766f8f33fa3f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:36.947569 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d04fd-6836-41af-9510-766f8f33fa3f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:36.949732 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d04fd-6836-41af-9510-766f8f33fa3f service swift] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:36.959290 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1397/2797] 10.222.0.27 () {60 vars in 1384 bytes} [Sun Sep 4 01:17:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 04 01:17:46.510080 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd8a75-12cd-4b61-9f56-4a85e35e5541 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:46.510631 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd8a75-12cd-4b61-9f56-4a85e35e5541 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:46.511130 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd8a75-12cd-4b61-9f56-4a85e35e5541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:46.530388 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-64cd8a75-12cd-4b61-9f56-4a85e35e5541 None None] MFA Rules not processed for user `35e7aa6333ae45d8af239fe22fc374f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:17:46.531713 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-64cd8a75-12cd-4b61-9f56-4a85e35e5541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:17:46.538510 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1401/2798] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:17:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:17:46.955623 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=96 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:17:47.271080 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d04fd-6836-41af-9510-766f8f33fa3f service swift] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:47.272727 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-892d04fd-6836-41af-9510-766f8f33fa3f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:17:47.361753 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d04fd-6836-41af-9510-766f8f33fa3f service swift] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:47.361753 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af63f29-aca7-4bf0-9e0e-ddbbf3411c66 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '018802a8d9c041119ad708430f885a89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:47.361753 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af63f29-aca7-4bf0-9e0e-ddbbf3411c66 service placement] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:47.361753 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af63f29-aca7-4bf0-9e0e-ddbbf3411c66 service placement] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:47.362335 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af63f29-aca7-4bf0-9e0e-ddbbf3411c66 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:47.362335 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9af63f29-aca7-4bf0-9e0e-ddbbf3411c66 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:17:47.369300 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af63f29-aca7-4bf0-9e0e-ddbbf3411c66 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:47.370926 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af63f29-aca7-4bf0-9e0e-ddbbf3411c66 service placement] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:47.380836 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1398/2799] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:17:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:17:50.277587 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812520a8-c965-4514-bf7a-a78713b8ae9d None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:50.278999 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812520a8-c965-4514-bf7a-a78713b8ae9d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:50.279491 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812520a8-c965-4514-bf7a-a78713b8ae9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:50.299732 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-812520a8-c965-4514-bf7a-a78713b8ae9d None None] MFA Rules not processed for user `d6d01ca673a1404c9b4cea51036c595c`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:17:50.300989 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-812520a8-c965-4514-bf7a-a78713b8ae9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:17:50.308071 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1402/2800] 10.222.0.27 () {58 vars in 1009 bytes} [Sun Sep 4 01:17:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:17:50.322319 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af63f29-aca7-4bf0-9e0e-ddbbf3411c66 service placement] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:17:50.333316 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af63f29-aca7-4bf0-9e0e-ddbbf3411c66 service placement] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:17:50.338607 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302bd6af-285e-43c7-9b08-5a2c376a15eb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2297d8b3fe8e47eb9fd39a6e3eea8f54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:17:50.339218 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302bd6af-285e-43c7-9b08-5a2c376a15eb service swift] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:17:50.339496 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302bd6af-285e-43c7-9b08-5a2c376a15eb service swift] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:17:50.339767 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302bd6af-285e-43c7-9b08-5a2c376a15eb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:17:50.350227 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302bd6af-285e-43c7-9b08-5a2c376a15eb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:17:50.352096 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302bd6af-285e-43c7-9b08-5a2c376a15eb service swift] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:17:50.361668 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1399/2801] 10.222.0.27 () {60 vars in 1384 bytes} [Sun Sep 4 01:17:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 04 01:18:00.294481 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=48 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:18:00.357862 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=38 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:18:03.231687 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812520a8-c965-4514-bf7a-a78713b8ae9d None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:18:03.234561 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-812520a8-c965-4514-bf7a-a78713b8ae9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:18:03.309936 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812520a8-c965-4514-bf7a-a78713b8ae9d None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:18:03.315534 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef079ea-64b0-484d-9bf1-3655ae6a70e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:18:03.316138 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef079ea-64b0-484d-9bf1-3655ae6a70e7 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:18:03.316419 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef079ea-64b0-484d-9bf1-3655ae6a70e7 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:18:03.316720 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef079ea-64b0-484d-9bf1-3655ae6a70e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:18:03.318425 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-6ef079ea-64b0-484d-9bf1-3655ae6a70e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:18:03.330596 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef079ea-64b0-484d-9bf1-3655ae6a70e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:18:03.332175 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef079ea-64b0-484d-9bf1-3655ae6a70e7 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:18:03.344048 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1403/2802] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:18:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 01:18:09.982869 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302bd6af-285e-43c7-9b08-5a2c376a15eb service swift] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:18:09.984435 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-302bd6af-285e-43c7-9b08-5a2c376a15eb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:18:09.999848 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302bd6af-285e-43c7-9b08-5a2c376a15eb service swift] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:18:10.005532 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302bfd4a-182d-420e-99f5-2759bf340f45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:18:10.006131 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302bfd4a-182d-420e-99f5-2759bf340f45 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:18:10.006417 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302bfd4a-182d-420e-99f5-2759bf340f45 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:18:10.006692 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302bfd4a-182d-420e-99f5-2759bf340f45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:18:10.008392 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-302bfd4a-182d-420e-99f5-2759bf340f45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:18:10.021169 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302bfd4a-182d-420e-99f5-2759bf340f45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:18:10.022779 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302bfd4a-182d-420e-99f5-2759bf340f45 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:18:10.034089 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1400/2803] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:18:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:18:13.338361 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=34 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:18:18.844890 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef079ea-64b0-484d-9bf1-3655ae6a70e7 service nova] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:18:18.854283 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef079ea-64b0-484d-9bf1-3655ae6a70e7 service nova] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:18:18.857886 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a50d57-8961-475b-94d1-b30704b6166e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:18:18.858482 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a50d57-8961-475b-94d1-b30704b6166e service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:18:18.862533 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a50d57-8961-475b-94d1-b30704b6166e service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:18:18.862533 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a50d57-8961-475b-94d1-b30704b6166e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:18:18.867868 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a50d57-8961-475b-94d1-b30704b6166e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:18:18.869326 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a50d57-8961-475b-94d1-b30704b6166e service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:18:18.878533 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1404/2804] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:18:18 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:18:20.029827 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=6 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:18:28.874450 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=6 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:20:38.897339 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302bfd4a-182d-420e-99f5-2759bf340f45 service nova] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:20:38.898761 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-302bfd4a-182d-420e-99f5-2759bf340f45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:20:38.912783 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302bfd4a-182d-420e-99f5-2759bf340f45 service nova] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:20:38.918274 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0591a5c-7ebd-4297-8cdb-9135d6236c49 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '018802a8d9c041119ad708430f885a89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:20:38.918863 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0591a5c-7ebd-4297-8cdb-9135d6236c49 service placement] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:20:38.919142 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0591a5c-7ebd-4297-8cdb-9135d6236c49 service placement] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:20:38.919413 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0591a5c-7ebd-4297-8cdb-9135d6236c49 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:20:38.921114 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e0591a5c-7ebd-4297-8cdb-9135d6236c49 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:20:38.932340 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0591a5c-7ebd-4297-8cdb-9135d6236c49 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:20:38.934635 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0591a5c-7ebd-4297-8cdb-9135d6236c49 service placement] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:20:38.944408 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1401/2805] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:20:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:20:41.262082 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a50d57-8961-475b-94d1-b30704b6166e service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:20:41.274742 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a50d57-8961-475b-94d1-b30704b6166e service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:20:41.281844 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c023f-90ab-4c2f-aae8-7a8d1b9eaac9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:20:41.282458 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c023f-90ab-4c2f-aae8-7a8d1b9eaac9 service nova] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:20:41.282740 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c023f-90ab-4c2f-aae8-7a8d1b9eaac9 service nova] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:20:41.283026 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c023f-90ab-4c2f-aae8-7a8d1b9eaac9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:20:41.284838 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c41c023f-90ab-4c2f-aae8-7a8d1b9eaac9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:20:41.298692 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c023f-90ab-4c2f-aae8-7a8d1b9eaac9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:20:41.300287 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c023f-90ab-4c2f-aae8-7a8d1b9eaac9 service nova] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:20:41.312825 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1405/2806] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:20:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:20:48.940196 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=6 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:20:51.308302 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=6 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:20:52.442085 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607fff9-5c91-4aeb-8691-a037ffb9635d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:20:52.442654 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607fff9-5c91-4aeb-8691-a037ffb9635d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:20:52.442654 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607fff9-5c91-4aeb-8691-a037ffb9635d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:20:52.465011 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-e607fff9-5c91-4aeb-8691-a037ffb9635d None None] MFA Rules not processed for user `c5819e21fbba41d3b235daf8f0d6769b`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:20:52.474935 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e607fff9-5c91-4aeb-8691-a037ffb9635d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:20:52.521275 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1402/2807] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:20:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:20:52.833430 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c023f-90ab-4c2f-aae8-7a8d1b9eaac9 service nova] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:20:52.843111 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c023f-90ab-4c2f-aae8-7a8d1b9eaac9 service nova] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:20:52.846730 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01718e00-daa1-498d-bc5a-f3beace9f189 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:20:52.847326 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01718e00-daa1-498d-bc5a-f3beace9f189 service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:20:52.847609 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01718e00-daa1-498d-bc5a-f3beace9f189 service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:20:52.847884 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01718e00-daa1-498d-bc5a-f3beace9f189 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:20:52.849674 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-01718e00-daa1-498d-bc5a-f3beace9f189 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:20:52.859007 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01718e00-daa1-498d-bc5a-f3beace9f189 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:20:52.860429 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01718e00-daa1-498d-bc5a-f3beace9f189 service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:20:52.868351 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1406/2808] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:20:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:20:53.762749 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3b3bb-da6f-4a0c-994a-384aef032335 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:20:53.763199 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3b3bb-da6f-4a0c-994a-384aef032335 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:20:53.763309 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3b3bb-da6f-4a0c-994a-384aef032335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:20:53.782701 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1ca3b3bb-da6f-4a0c-994a-384aef032335 None None] MFA Rules not processed for user `d6d01ca673a1404c9b4cea51036c595c`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:20:53.783900 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1ca3b3bb-da6f-4a0c-994a-384aef032335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:20:53.791080 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1403/2809] 10.222.0.27 () {58 vars in 967 bytes} [Sun Sep 4 01:20:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:20:53.830349 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01718e00-daa1-498d-bc5a-f3beace9f189 service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:20:53.842846 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01718e00-daa1-498d-bc5a-f3beace9f189 service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:20:53.847753 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a693e562-cea6-41c3-8bc3-35944523c444 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2297d8b3fe8e47eb9fd39a6e3eea8f54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:20:53.848344 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a693e562-cea6-41c3-8bc3-35944523c444 service swift] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:20:53.848652 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a693e562-cea6-41c3-8bc3-35944523c444 service swift] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:20:53.848927 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a693e562-cea6-41c3-8bc3-35944523c444 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:20:53.857481 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a693e562-cea6-41c3-8bc3-35944523c444 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:20:53.859045 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a693e562-cea6-41c3-8bc3-35944523c444 service swift] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:20:53.866994 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1407/2810] 10.222.0.27 () {60 vars in 1384 bytes} [Sun Sep 4 01:20:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 04 01:20:58.464214 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca3b3bb-da6f-4a0c-994a-384aef032335 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:20:58.477273 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca3b3bb-da6f-4a0c-994a-384aef032335 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:20:58.483309 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da56f1a3-3b18-4d24-bc92-571f75959cbc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '018802a8d9c041119ad708430f885a89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:20:58.485991 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56f1a3-3b18-4d24-bc92-571f75959cbc service placement] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:20:58.485991 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56f1a3-3b18-4d24-bc92-571f75959cbc service placement] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:20:58.485991 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56f1a3-3b18-4d24-bc92-571f75959cbc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:20:58.486324 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-da56f1a3-3b18-4d24-bc92-571f75959cbc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:20:58.499408 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da56f1a3-3b18-4d24-bc92-571f75959cbc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:20:58.501017 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da56f1a3-3b18-4d24-bc92-571f75959cbc service placement] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:20:58.512062 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1404/2811] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:20:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:21:02.843403 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a693e562-cea6-41c3-8bc3-35944523c444 service swift] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:21:02.855885 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a693e562-cea6-41c3-8bc3-35944523c444 service swift] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:21:02.861265 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562192d-0fde-4e3a-9976-29556a046032 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '018802a8d9c041119ad708430f885a89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:21:02.861871 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562192d-0fde-4e3a-9976-29556a046032 service placement] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:21:02.862155 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562192d-0fde-4e3a-9976-29556a046032 service placement] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:21:02.862431 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562192d-0fde-4e3a-9976-29556a046032 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:21:02.864222 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5562192d-0fde-4e3a-9976-29556a046032 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:21:02.883789 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5562192d-0fde-4e3a-9976-29556a046032 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:21:02.885537 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5562192d-0fde-4e3a-9976-29556a046032 service placement] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:21:02.896310 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1408/2812] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:21:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:21:03.050749 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da56f1a3-3b18-4d24-bc92-571f75959cbc service placement] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:21:03.116666 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da56f1a3-3b18-4d24-bc92-571f75959cbc service placement] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:21:03.121577 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5185c850-9b8e-400a-a66a-eac777bc8976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:21:03.122203 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185c850-9b8e-400a-a66a-eac777bc8976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:21:03.122485 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185c850-9b8e-400a-a66a-eac777bc8976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:21:03.122760 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185c850-9b8e-400a-a66a-eac777bc8976 admin admin] PATH_INFO: `/v3/users/35e7aa6333ae45d8af239fe22fc374f2` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:21:03.123870 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5185c850-9b8e-400a-a66a-eac777bc8976 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35e7aa6333ae45d8af239fe22fc374f2)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:21:03.125856 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5185c850-9b8e-400a-a66a-eac777bc8976 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:21:03.354872 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5185c850-9b8e-400a-a66a-eac777bc8976 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35e7aa6333ae45d8af239fe22fc374f2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:21:03.357642 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5185c850-9b8e-400a-a66a-eac777bc8976 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35e7aa6333ae45d8af239fe22fc374f2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:21:03.373008 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5185c850-9b8e-400a-a66a-eac777bc8976 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35e7aa6333ae45d8af239fe22fc374f2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:21:03.440143 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5185c850-9b8e-400a-a66a-eac777bc8976 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35e7aa6333ae45d8af239fe22fc374f2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:21:03.440795 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5185c850-9b8e-400a-a66a-eac777bc8976 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35e7aa6333ae45d8af239fe22fc374f2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:21:03.460032 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1405/2813] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:21:03 2022] DELETE /identity/v3/users/35e7aa6333ae45d8af239fe22fc374f2 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:21:03.757800 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562192d-0fde-4e3a-9976-29556a046032 service placement] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:21:03.775991 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562192d-0fde-4e3a-9976-29556a046032 service placement] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:21:03.837941 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe432fe-918d-4713-a4d1-94ea51424171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:21:03.838517 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe432fe-918d-4713-a4d1-94ea51424171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:21:03.838796 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe432fe-918d-4713-a4d1-94ea51424171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:21:03.839068 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe432fe-918d-4713-a4d1-94ea51424171 admin admin] PATH_INFO: `/v3/projects/5a4534146f2f4d5f8f2ed8a2b9d1fb3e` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:21:03.840213 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe432fe-918d-4713-a4d1-94ea51424171 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a4534146f2f4d5f8f2ed8a2b9d1fb3e)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:21:03.842157 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe432fe-918d-4713-a4d1-94ea51424171 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:21:03.946822 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-5fe432fe-918d-4713-a4d1-94ea51424171 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a4534146f2f4d5f8f2ed8a2b9d1fb3e', 'request_id': 'req-5fe432fe-918d-4713-a4d1-94ea51424171'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:21:04.017143 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-5fe432fe-918d-4713-a4d1-94ea51424171 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a4534146f2f4d5f8f2ed8a2b9d1fb3e', 'request_id': 'req-5fe432fe-918d-4713-a4d1-94ea51424171'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:21:04.022945 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-5fe432fe-918d-4713-a4d1-94ea51424171 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a4534146f2f4d5f8f2ed8a2b9d1fb3e', 'request_id': 'req-5fe432fe-918d-4713-a4d1-94ea51424171'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:21:04.034343 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-5fe432fe-918d-4713-a4d1-94ea51424171 admin admin] The token cache is being invalidate because project 5a4534146f2f4d5f8f2ed8a2b9d1fb3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:21:04.034647 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-5fe432fe-918d-4713-a4d1-94ea51424171 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:21:04.035265 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-5fe432fe-918d-4713-a4d1-94ea51424171 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:21:04.036725 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1409/2814] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:21:03 2022] DELETE /identity/v3/projects/5a4534146f2f4d5f8f2ed8a2b9d1fb3e => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:21:04.040809 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5185c850-9b8e-400a-a66a-eac777bc8976 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:21:04.043012 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5185c850-9b8e-400a-a66a-eac777bc8976 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:21:04.052355 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:21:04.056665 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5185c850-9b8e-400a-a66a-eac777bc8976 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:21:04.112690 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d968fb-2bf3-490c-bac8-2d431486c252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:21:04.113233 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d968fb-2bf3-490c-bac8-2d431486c252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:21:04.113512 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d968fb-2bf3-490c-bac8-2d431486c252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:21:04.113782 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d968fb-2bf3-490c-bac8-2d431486c252 admin admin] PATH_INFO: `/v3/users/7a727c0cabe748c09c942c22abfbb7f2` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:21:04.114898 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d968fb-2bf3-490c-bac8-2d431486c252 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a727c0cabe748c09c942c22abfbb7f2)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:21:04.116837 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d968fb-2bf3-490c-bac8-2d431486c252 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:21:04.339580 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-02d968fb-2bf3-490c-bac8-2d431486c252 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a727c0cabe748c09c942c22abfbb7f2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:21:04.340251 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-02d968fb-2bf3-490c-bac8-2d431486c252 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a727c0cabe748c09c942c22abfbb7f2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:21:04.347140 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-02d968fb-2bf3-490c-bac8-2d431486c252 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a727c0cabe748c09c942c22abfbb7f2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:21:04.431594 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-02d968fb-2bf3-490c-bac8-2d431486c252 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a727c0cabe748c09c942c22abfbb7f2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:21:04.432132 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-02d968fb-2bf3-490c-bac8-2d431486c252 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a727c0cabe748c09c942c22abfbb7f2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:21:04.450511 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1406/2815] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:21:04 2022] DELETE /identity/v3/users/7a727c0cabe748c09c942c22abfbb7f2 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:21:04.705158 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe432fe-918d-4713-a4d1-94ea51424171 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:21:04.706648 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5fe432fe-918d-4713-a4d1-94ea51424171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:21:04.720722 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe432fe-918d-4713-a4d1-94ea51424171 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:21:04.770710 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:21:04.780224 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c26388-e3de-477f-9952-5e9ba94edc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:21:04.780884 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c26388-e3de-477f-9952-5e9ba94edc72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:21:04.781177 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c26388-e3de-477f-9952-5e9ba94edc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:21:04.781437 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c26388-e3de-477f-9952-5e9ba94edc72 admin admin] PATH_INFO: `/v3/projects/a4819545ee3341588c2e82e386e89582` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:21:04.782543 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c26388-e3de-477f-9952-5e9ba94edc72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4819545ee3341588c2e82e386e89582)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:21:04.784445 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c26388-e3de-477f-9952-5e9ba94edc72 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:21:04.886462 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-02c26388-e3de-477f-9952-5e9ba94edc72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4819545ee3341588c2e82e386e89582', 'request_id': 'req-02c26388-e3de-477f-9952-5e9ba94edc72'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:21:04.959928 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-02c26388-e3de-477f-9952-5e9ba94edc72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4819545ee3341588c2e82e386e89582', 'request_id': 'req-02c26388-e3de-477f-9952-5e9ba94edc72'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:21:04.968853 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-02c26388-e3de-477f-9952-5e9ba94edc72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4819545ee3341588c2e82e386e89582', 'request_id': 'req-02c26388-e3de-477f-9952-5e9ba94edc72'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:21:04.981040 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-02c26388-e3de-477f-9952-5e9ba94edc72 admin admin] The token cache is being invalidate because project a4819545ee3341588c2e82e386e89582 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:21:04.981326 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-02c26388-e3de-477f-9952-5e9ba94edc72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:21:04.982272 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-02c26388-e3de-477f-9952-5e9ba94edc72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:21:04.983734 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1410/2816] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:21:04 2022] DELETE /identity/v3/projects/a4819545ee3341588c2e82e386e89582 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:21:14.446179 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=51,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:21:14.977748 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=28,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:22:28.297882 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d968fb-2bf3-490c-bac8-2d431486c252 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:22:28.299538 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-02d968fb-2bf3-490c-bac8-2d431486c252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:22:28.374390 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d968fb-2bf3-490c-bac8-2d431486c252 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:22:28.440548 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53571c12-6dc2-4a5f-a678-1730a89118a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68bc00bda5194cd8a7928f8c4df4ccd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:22:28.441169 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53571c12-6dc2-4a5f-a678-1730a89118a3 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:22:28.441450 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53571c12-6dc2-4a5f-a678-1730a89118a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:22:28.441726 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53571c12-6dc2-4a5f-a678-1730a89118a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:22:28.443461 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-53571c12-6dc2-4a5f-a678-1730a89118a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:22:28.503544 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53571c12-6dc2-4a5f-a678-1730a89118a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:22:28.505168 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53571c12-6dc2-4a5f-a678-1730a89118a3 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:22:28.514638 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1407/2817] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:22:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:22:30.203578 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c26388-e3de-477f-9952-5e9ba94edc72 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:22:30.205086 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-02c26388-e3de-477f-9952-5e9ba94edc72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:22:30.264195 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c26388-e3de-477f-9952-5e9ba94edc72 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:22:30.267821 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df69de03-cf98-4517-bd83-b5df97100119 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:22:30.268421 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69de03-cf98-4517-bd83-b5df97100119 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:22:30.268737 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69de03-cf98-4517-bd83-b5df97100119 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:22:30.269012 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69de03-cf98-4517-bd83-b5df97100119 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:22:30.270711 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-df69de03-cf98-4517-bd83-b5df97100119 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:22:30.284751 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df69de03-cf98-4517-bd83-b5df97100119 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:22:30.286167 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df69de03-cf98-4517-bd83-b5df97100119 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:22:30.297810 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1411/2818] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:22:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:22:35.010397 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53571c12-6dc2-4a5f-a678-1730a89118a3 service nova] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:22:35.020549 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53571c12-6dc2-4a5f-a678-1730a89118a3 service nova] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:22:35.024059 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ecdf11-83a8-45eb-9bb3-26afc7df4d74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:22:35.024691 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ecdf11-83a8-45eb-9bb3-26afc7df4d74 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:22:35.024976 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ecdf11-83a8-45eb-9bb3-26afc7df4d74 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:22:35.025253 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ecdf11-83a8-45eb-9bb3-26afc7df4d74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:22:35.026998 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-87ecdf11-83a8-45eb-9bb3-26afc7df4d74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:22:35.040902 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ecdf11-83a8-45eb-9bb3-26afc7df4d74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:22:35.042322 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ecdf11-83a8-45eb-9bb3-26afc7df4d74 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:22:35.053292 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1408/2819] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:22:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:22:37.028125 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df69de03-cf98-4517-bd83-b5df97100119 service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:22:37.029688 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-df69de03-cf98-4517-bd83-b5df97100119 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:22:37.086816 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df69de03-cf98-4517-bd83-b5df97100119 service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:22:37.090569 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b988e7fd-c0fa-4e6c-b381-8a57fa84d77c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:22:37.091180 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b988e7fd-c0fa-4e6c-b381-8a57fa84d77c service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:22:37.091438 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b988e7fd-c0fa-4e6c-b381-8a57fa84d77c service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:22:37.091906 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b988e7fd-c0fa-4e6c-b381-8a57fa84d77c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:22:37.093669 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b988e7fd-c0fa-4e6c-b381-8a57fa84d77c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:22:37.103313 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b988e7fd-c0fa-4e6c-b381-8a57fa84d77c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:22:37.104766 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b988e7fd-c0fa-4e6c-b381-8a57fa84d77c service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:22:37.112992 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1412/2820] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:22:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:22:45.048381 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=90 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:22:47.108996 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=52 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:22:54.417343 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ecdf11-83a8-45eb-9bb3-26afc7df4d74 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:22:54.418721 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-87ecdf11-83a8-45eb-9bb3-26afc7df4d74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:22:54.430690 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ecdf11-83a8-45eb-9bb3-26afc7df4d74 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:22:54.434198 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b850e1af-6518-4a29-977b-4a035af4945e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:22:54.434782 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850e1af-6518-4a29-977b-4a035af4945e service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:22:54.435056 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850e1af-6518-4a29-977b-4a035af4945e service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:22:54.435323 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850e1af-6518-4a29-977b-4a035af4945e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:22:54.436999 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b850e1af-6518-4a29-977b-4a035af4945e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:22:54.498934 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b850e1af-6518-4a29-977b-4a035af4945e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:22:54.500359 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b850e1af-6518-4a29-977b-4a035af4945e service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:22:54.510682 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1409/2821] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:22:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:22:58.111786 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b988e7fd-c0fa-4e6c-b381-8a57fa84d77c service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:22:58.113889 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b988e7fd-c0fa-4e6c-b381-8a57fa84d77c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:22:58.128763 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b988e7fd-c0fa-4e6c-b381-8a57fa84d77c service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:22:58.133881 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b26ecb-cc01-4aa5-8254-b3afb14ccc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:22:58.134509 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b26ecb-cc01-4aa5-8254-b3afb14ccc65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:22:58.136362 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b26ecb-cc01-4aa5-8254-b3afb14ccc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:22:58.136362 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b26ecb-cc01-4aa5-8254-b3afb14ccc65 admin admin] PATH_INFO: `/v3/users/c5819e21fbba41d3b235daf8f0d6769b` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:22:58.136362 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b26ecb-cc01-4aa5-8254-b3afb14ccc65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5819e21fbba41d3b235daf8f0d6769b)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:22:58.138099 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b26ecb-cc01-4aa5-8254-b3afb14ccc65 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:22:58.358833 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f0b26ecb-cc01-4aa5-8254-b3afb14ccc65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5819e21fbba41d3b235daf8f0d6769b'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:22:58.359442 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f0b26ecb-cc01-4aa5-8254-b3afb14ccc65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5819e21fbba41d3b235daf8f0d6769b'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:22:58.453095 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f0b26ecb-cc01-4aa5-8254-b3afb14ccc65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5819e21fbba41d3b235daf8f0d6769b'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:22:58.453741 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f0b26ecb-cc01-4aa5-8254-b3afb14ccc65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5819e21fbba41d3b235daf8f0d6769b'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:22:58.466856 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-f0b26ecb-cc01-4aa5-8254-b3afb14ccc65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5819e21fbba41d3b235daf8f0d6769b'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:22:58.479602 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1413/2822] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:22:58 2022] DELETE /identity/v3/users/c5819e21fbba41d3b235daf8f0d6769b => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:22:58.709651 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b850e1af-6518-4a29-977b-4a035af4945e service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:22:58.719613 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b850e1af-6518-4a29-977b-4a035af4945e service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:22:58.776115 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eed3fd-2ef3-462c-9e88-295546a0c6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:22:58.776785 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eed3fd-2ef3-462c-9e88-295546a0c6c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:22:58.777070 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eed3fd-2ef3-462c-9e88-295546a0c6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:22:58.777363 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eed3fd-2ef3-462c-9e88-295546a0c6c5 admin admin] PATH_INFO: `/v3/projects/5cfeb662f87744b195af64361774c98a` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:22:58.778475 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eed3fd-2ef3-462c-9e88-295546a0c6c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cfeb662f87744b195af64361774c98a)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:22:58.780379 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eed3fd-2ef3-462c-9e88-295546a0c6c5 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:22:58.885310 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-66eed3fd-2ef3-462c-9e88-295546a0c6c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cfeb662f87744b195af64361774c98a', 'request_id': 'req-66eed3fd-2ef3-462c-9e88-295546a0c6c5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:22:58.953164 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-66eed3fd-2ef3-462c-9e88-295546a0c6c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cfeb662f87744b195af64361774c98a', 'request_id': 'req-66eed3fd-2ef3-462c-9e88-295546a0c6c5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:22:58.959299 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-66eed3fd-2ef3-462c-9e88-295546a0c6c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cfeb662f87744b195af64361774c98a', 'request_id': 'req-66eed3fd-2ef3-462c-9e88-295546a0c6c5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:22:58.970285 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-66eed3fd-2ef3-462c-9e88-295546a0c6c5 admin admin] The token cache is being invalidate because project 5cfeb662f87744b195af64361774c98a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:22:58.970572 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-66eed3fd-2ef3-462c-9e88-295546a0c6c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:22:58.971198 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-66eed3fd-2ef3-462c-9e88-295546a0c6c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:22:58.973346 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1410/2823] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:22:58 2022] DELETE /identity/v3/projects/5cfeb662f87744b195af64361774c98a => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:22:58.978315 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a7cdf-61ad-4f68-8740-2016b7e62257 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:22:58.978605 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a7cdf-61ad-4f68-8740-2016b7e62257 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:22:58.978887 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a7cdf-61ad-4f68-8740-2016b7e62257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:22:58.998490 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-224a7cdf-61ad-4f68-8740-2016b7e62257 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:22:58.999689 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-224a7cdf-61ad-4f68-8740-2016b7e62257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:22:59.058049 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1414/2824] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:22:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:22:59.067284 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc7b10-0938-4a9f-a66f-46bc5e102da1 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:22:59.067568 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc7b10-0938-4a9f-a66f-46bc5e102da1 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:22:59.067840 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc7b10-0938-4a9f-a66f-46bc5e102da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:22:59.087800 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-ecfc7b10-0938-4a9f-a66f-46bc5e102da1 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:22:59.089049 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ecfc7b10-0938-4a9f-a66f-46bc5e102da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:22:59.096955 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1411/2825] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:22:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:22:59.101178 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a7cdf-61ad-4f68-8740-2016b7e62257 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:22:59.110758 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a7cdf-61ad-4f68-8740-2016b7e62257 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:22:59.115936 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef81bb-5c8d-4f00-8d2a-75ab2bcba7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:22:59.117107 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef81bb-5c8d-4f00-8d2a-75ab2bcba7da admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:22:59.117435 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef81bb-5c8d-4f00-8d2a-75ab2bcba7da admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:22:59.117715 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef81bb-5c8d-4f00-8d2a-75ab2bcba7da admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:22:59.118391 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ef81bb-5c8d-4f00-8d2a-75ab2bcba7da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:22:59.118700 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ef81bb-5c8d-4f00-8d2a-75ab2bcba7da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:22:59.120307 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ef81bb-5c8d-4f00-8d2a-75ab2bcba7da admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:22:59.132522 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1415/2826] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:22:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:22:59.136664 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc7b10-0938-4a9f-a66f-46bc5e102da1 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:22:59.146013 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc7b10-0938-4a9f-a66f-46bc5e102da1 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:22:59.150633 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3707aa-b330-47d1-bb4e-75c218c2a6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:22:59.151249 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3707aa-b330-47d1-bb4e-75c218c2a6ac admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:22:59.151524 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3707aa-b330-47d1-bb4e-75c218c2a6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:22:59.151790 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3707aa-b330-47d1-bb4e-75c218c2a6ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:22:59.152412 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3707aa-b330-47d1-bb4e-75c218c2a6ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:22:59.154296 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3707aa-b330-47d1-bb4e-75c218c2a6ac admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:22:59.169502 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:22:59.271464 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1412/2827] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:22:59 2022] POST /identity/v3/projects => generated 379 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:22:59.275719 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef81bb-5c8d-4f00-8d2a-75ab2bcba7da admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:22:59.286359 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef81bb-5c8d-4f00-8d2a-75ab2bcba7da admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:22:59.342347 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a654fb-d4c5-486c-bcbb-775078cfc29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:22:59.342941 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a654fb-d4c5-486c-bcbb-775078cfc29e admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:22:59.343218 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a654fb-d4c5-486c-bcbb-775078cfc29e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:22:59.343497 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a654fb-d4c5-486c-bcbb-775078cfc29e admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:22:59.344110 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a654fb-d4c5-486c-bcbb-775078cfc29e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:22:59.346034 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a654fb-d4c5-486c-bcbb-775078cfc29e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:22:59.354538 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:22:59.452763 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1416/2828] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:22:59 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:22:59.457076 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3707aa-b330-47d1-bb4e-75c218c2a6ac admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:22:59.467232 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3707aa-b330-47d1-bb4e-75c218c2a6ac admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:22:59.471785 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d071654e-742b-4178-ba6c-35b5ce747d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:22:59.472922 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071654e-742b-4178-ba6c-35b5ce747d95 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:22:59.473199 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071654e-742b-4178-ba6c-35b5ce747d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:22:59.473508 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071654e-742b-4178-ba6c-35b5ce747d95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:22:59.474195 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d071654e-742b-4178-ba6c-35b5ce747d95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:22:59.475756 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d071654e-742b-4178-ba6c-35b5ce747d95 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:22:59.488053 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1413/2829] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:22:59.491356 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a654fb-d4c5-486c-bcbb-775078cfc29e admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:22:59.501316 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a654fb-d4c5-486c-bcbb-775078cfc29e admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:22:59.506559 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3139e2dc-4d2f-432c-899b-38b101031a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:22:59.507349 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139e2dc-4d2f-432c-899b-38b101031a67 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:22:59.507857 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139e2dc-4d2f-432c-899b-38b101031a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:22:59.508394 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139e2dc-4d2f-432c-899b-38b101031a67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:22:59.509092 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3139e2dc-4d2f-432c-899b-38b101031a67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:22:59.510705 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3139e2dc-4d2f-432c-899b-38b101031a67 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:22:59.521308 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1417/2830] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:22:59.525170 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d071654e-742b-4178-ba6c-35b5ce747d95 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:22:59.535422 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d071654e-742b-4178-ba6c-35b5ce747d95 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:22:59.540037 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e22b5-c76d-4f3d-b03b-cea6afc2d26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:22:59.540706 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e22b5-c76d-4f3d-b03b-cea6afc2d26e admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:22:59.540989 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e22b5-c76d-4f3d-b03b-cea6afc2d26e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:22:59.541262 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e22b5-c76d-4f3d-b03b-cea6afc2d26e admin admin] PATH_INFO: `/v3/projects/2e95ad0e5c9946e0bc767bbfd85009bc/users/4e255d416fed46afbdb9d706de70056b/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:22:59.556299 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926e22b5-c76d-4f3d-b03b-cea6afc2d26e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e95ad0e5c9946e0bc767bbfd85009bc, user_id=4e255d416fed46afbdb9d706de70056b, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:22:59.558975 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926e22b5-c76d-4f3d-b03b-cea6afc2d26e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:22:59.651332 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1414/2831] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:22:59 2022] PUT /identity/v3/projects/2e95ad0e5c9946e0bc767bbfd85009bc/users/4e255d416fed46afbdb9d706de70056b/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:22:59.662374 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d1197-4f32-4617-8b62-963c7186e9d1 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:22:59.662665 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d1197-4f32-4617-8b62-963c7186e9d1 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:22:59.662946 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d1197-4f32-4617-8b62-963c7186e9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:22:59.690447 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-ac9d1197-4f32-4617-8b62-963c7186e9d1 None None] MFA Rules not processed for user `4e255d416fed46afbdb9d706de70056b`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:22:59.691696 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-ac9d1197-4f32-4617-8b62-963c7186e9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:22:59.757495 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1418/2832] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:22:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:22:59.762002 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e22b5-c76d-4f3d-b03b-cea6afc2d26e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:22:59.823882 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e22b5-c76d-4f3d-b03b-cea6afc2d26e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:22:59.828521 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f2ea9-72e5-4c43-9efe-d4d14d7d1579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:22:59.829129 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f2ea9-72e5-4c43-9efe-d4d14d7d1579 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:22:59.829411 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f2ea9-72e5-4c43-9efe-d4d14d7d1579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:22:59.829688 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f2ea9-72e5-4c43-9efe-d4d14d7d1579 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:22:59.830327 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5f2ea9-72e5-4c43-9efe-d4d14d7d1579 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:22:59.832763 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5f2ea9-72e5-4c43-9efe-d4d14d7d1579 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:22:59.945656 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1415/2833] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:22:59 2022] POST /identity/v3/projects => generated 379 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:22:59.949717 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d1197-4f32-4617-8b62-963c7186e9d1 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:22:59.961475 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d1197-4f32-4617-8b62-963c7186e9d1 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:00.017458 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1bd69e-6122-4a7d-8074-256a98e9c47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:00.017979 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bd69e-6122-4a7d-8074-256a98e9c47f admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:00.018257 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bd69e-6122-4a7d-8074-256a98e9c47f admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:00.018528 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bd69e-6122-4a7d-8074-256a98e9c47f admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:00.019144 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1bd69e-6122-4a7d-8074-256a98e9c47f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:00.021631 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1bd69e-6122-4a7d-8074-256a98e9c47f admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:00.134174 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1419/2834] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:22:59 2022] POST /identity/v3/users => generated 296 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:23:00.138282 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f2ea9-72e5-4c43-9efe-d4d14d7d1579 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:00.149133 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f2ea9-72e5-4c43-9efe-d4d14d7d1579 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:00.153640 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347795d-99fd-42b1-8afa-e80272593a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:00.154228 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347795d-99fd-42b1-8afa-e80272593a3d admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:00.154585 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347795d-99fd-42b1-8afa-e80272593a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:00.154814 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347795d-99fd-42b1-8afa-e80272593a3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:00.155479 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f347795d-99fd-42b1-8afa-e80272593a3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:00.157082 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f347795d-99fd-42b1-8afa-e80272593a3d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:00.167883 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1416/2835] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:23:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:23:00.171974 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1bd69e-6122-4a7d-8074-256a98e9c47f admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:00.182178 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1bd69e-6122-4a7d-8074-256a98e9c47f admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:00.186836 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d4cac6-3b33-4a8a-b431-87f7489b9ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:00.187477 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4cac6-3b33-4a8a-b431-87f7489b9ce0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:00.190774 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4cac6-3b33-4a8a-b431-87f7489b9ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:00.190774 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4cac6-3b33-4a8a-b431-87f7489b9ce0 admin admin] PATH_INFO: `/v3/projects/1ce281cf7fb84d0e834aed98d0b7c4b9/users/435bfca952554cbf829bd54e07ba7d13/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:00.203710 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d4cac6-3b33-4a8a-b431-87f7489b9ce0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ce281cf7fb84d0e834aed98d0b7c4b9, user_id=435bfca952554cbf829bd54e07ba7d13, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:00.206456 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d4cac6-3b33-4a8a-b431-87f7489b9ce0 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:00.299303 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1420/2836] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:23:00 2022] PUT /identity/v3/projects/1ce281cf7fb84d0e834aed98d0b7c4b9/users/435bfca952554cbf829bd54e07ba7d13/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:23:00.303427 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347795d-99fd-42b1-8afa-e80272593a3d admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:00.313372 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347795d-99fd-42b1-8afa-e80272593a3d admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:00.370354 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a6be7-4e6d-4c2f-8505-64608e789345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:00.370955 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a6be7-4e6d-4c2f-8505-64608e789345 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:00.371239 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a6be7-4e6d-4c2f-8505-64608e789345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:00.371514 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a6be7-4e6d-4c2f-8505-64608e789345 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:00.372147 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046a6be7-4e6d-4c2f-8505-64608e789345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:00.373773 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046a6be7-4e6d-4c2f-8505-64608e789345 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:00.384544 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1417/2837] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:23:00 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:23:00.388692 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d4cac6-3b33-4a8a-b431-87f7489b9ce0 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:00.399445 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d4cac6-3b33-4a8a-b431-87f7489b9ce0 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:00.404136 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7134cb63-e24f-411b-aef6-01874c1c5f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:00.404809 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134cb63-e24f-411b-aef6-01874c1c5f6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:00.405088 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134cb63-e24f-411b-aef6-01874c1c5f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:00.405364 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134cb63-e24f-411b-aef6-01874c1c5f6e admin admin] PATH_INFO: `/v3/projects/1ce281cf7fb84d0e834aed98d0b7c4b9/users/435bfca952554cbf829bd54e07ba7d13/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:00.407269 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7134cb63-e24f-411b-aef6-01874c1c5f6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ce281cf7fb84d0e834aed98d0b7c4b9, user_id=435bfca952554cbf829bd54e07ba7d13, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:00.410044 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7134cb63-e24f-411b-aef6-01874c1c5f6e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:00.420112 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1421/2838] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:23:00 2022] PUT /identity/v3/projects/1ce281cf7fb84d0e834aed98d0b7c4b9/users/435bfca952554cbf829bd54e07ba7d13/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:23:00.431947 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db39ad-8e1b-4085-8752-dfdde8ff62ce None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:00.432239 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db39ad-8e1b-4085-8752-dfdde8ff62ce None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:00.432543 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db39ad-8e1b-4085-8752-dfdde8ff62ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:00.460716 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-b4db39ad-8e1b-4085-8752-dfdde8ff62ce None None] MFA Rules not processed for user `435bfca952554cbf829bd54e07ba7d13`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:23:00.461933 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b4db39ad-8e1b-4085-8752-dfdde8ff62ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:00.479993 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:23:00.530019 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1418/2839] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:23:00 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:23:00.541347 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7134cb63-e24f-411b-aef6-01874c1c5f6e admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:00.545743 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-7134cb63-e24f-411b-aef6-01874c1c5f6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:00.570505 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:23:00.601079 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7134cb63-e24f-411b-aef6-01874c1c5f6e admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:00.604771 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8652d3-5a89-42a4-b0be-b36b0a457e82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:00.605368 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8652d3-5a89-42a4-b0be-b36b0a457e82 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:00.605649 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8652d3-5a89-42a4-b0be-b36b0a457e82 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:00.605963 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8652d3-5a89-42a4-b0be-b36b0a457e82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:00.615361 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8652d3-5a89-42a4-b0be-b36b0a457e82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:00.616858 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8652d3-5a89-42a4-b0be-b36b0a457e82 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:00.626085 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1422/2840] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:23:00 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:23:00.859248 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db39ad-8e1b-4085-8752-dfdde8ff62ce None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:00.870572 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db39ad-8e1b-4085-8752-dfdde8ff62ce None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:00.874197 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e5481-a152-464a-a76f-ebf296592bcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:00.874790 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e5481-a152-464a-a76f-ebf296592bcd service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:00.875106 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e5481-a152-464a-a76f-ebf296592bcd service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:00.875367 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e5481-a152-464a-a76f-ebf296592bcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:00.933083 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961e5481-a152-464a-a76f-ebf296592bcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:00.934511 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961e5481-a152-464a-a76f-ebf296592bcd service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:00.943639 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1419/2841] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:23:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:23:09.451700 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8652d3-5a89-42a4-b0be-b36b0a457e82 service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:09.513439 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8652d3-5a89-42a4-b0be-b36b0a457e82 service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:09.517876 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d032852-38e9-4695-9e8b-e6d255a20b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:09.518486 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d032852-38e9-4695-9e8b-e6d255a20b10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:09.518762 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d032852-38e9-4695-9e8b-e6d255a20b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:09.519034 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d032852-38e9-4695-9e8b-e6d255a20b10 admin admin] PATH_INFO: `/v3/users/4e255d416fed46afbdb9d706de70056b` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:09.520119 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d032852-38e9-4695-9e8b-e6d255a20b10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e255d416fed46afbdb9d706de70056b)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:09.522045 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d032852-38e9-4695-9e8b-e6d255a20b10 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:09.675461 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-5d032852-38e9-4695-9e8b-e6d255a20b10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e255d416fed46afbdb9d706de70056b'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:09.676098 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-5d032852-38e9-4695-9e8b-e6d255a20b10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e255d416fed46afbdb9d706de70056b'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:09.747775 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-5d032852-38e9-4695-9e8b-e6d255a20b10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e255d416fed46afbdb9d706de70056b'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:09.748384 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-5d032852-38e9-4695-9e8b-e6d255a20b10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e255d416fed46afbdb9d706de70056b'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:09.760821 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-5d032852-38e9-4695-9e8b-e6d255a20b10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e255d416fed46afbdb9d706de70056b'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:09.772770 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1423/2842] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:23:09 2022] DELETE /identity/v3/users/4e255d416fed46afbdb9d706de70056b => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:23:09.777428 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d05bb-6cde-4040-ba02-6cfd0fe47aa5 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:09.777938 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d05bb-6cde-4040-ba02-6cfd0fe47aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:09.778051 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d05bb-6cde-4040-ba02-6cfd0fe47aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:09.798022 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-810d05bb-6cde-4040-ba02-6cfd0fe47aa5 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:23:09.806669 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-810d05bb-6cde-4040-ba02-6cfd0fe47aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:09.859117 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1420/2843] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:23:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:23:09.869706 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d032852-38e9-4695-9e8b-e6d255a20b10 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:09.871088 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5d032852-38e9-4695-9e8b-e6d255a20b10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:09.916124 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:23:09.927440 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d032852-38e9-4695-9e8b-e6d255a20b10 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:09.931646 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fab823-8d8f-4cd3-8f5e-854cb89c8cbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:09.932245 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fab823-8d8f-4cd3-8f5e-854cb89c8cbc service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:09.932620 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fab823-8d8f-4cd3-8f5e-854cb89c8cbc service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:09.932895 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fab823-8d8f-4cd3-8f5e-854cb89c8cbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:09.941694 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fab823-8d8f-4cd3-8f5e-854cb89c8cbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:09.943107 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fab823-8d8f-4cd3-8f5e-854cb89c8cbc service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:09.952672 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1424/2844] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:23:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:23:10.383695 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d05bb-6cde-4040-ba02-6cfd0fe47aa5 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:10.386349 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-810d05bb-6cde-4040-ba02-6cfd0fe47aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:10.398942 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d05bb-6cde-4040-ba02-6cfd0fe47aa5 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:10.404726 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9285019-d124-45ee-affb-3fe2558b7927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:10.405361 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9285019-d124-45ee-affb-3fe2558b7927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:10.405644 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9285019-d124-45ee-affb-3fe2558b7927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:10.405921 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9285019-d124-45ee-affb-3fe2558b7927 admin admin] PATH_INFO: `/v3/projects/2e95ad0e5c9946e0bc767bbfd85009bc` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:10.407055 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9285019-d124-45ee-affb-3fe2558b7927 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e95ad0e5c9946e0bc767bbfd85009bc)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:10.408992 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9285019-d124-45ee-affb-3fe2558b7927 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:10.521428 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b9285019-d124-45ee-affb-3fe2558b7927 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e95ad0e5c9946e0bc767bbfd85009bc', 'request_id': 'req-b9285019-d124-45ee-affb-3fe2558b7927'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:10.523048 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:23:10.618519 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b9285019-d124-45ee-affb-3fe2558b7927 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e95ad0e5c9946e0bc767bbfd85009bc', 'request_id': 'req-b9285019-d124-45ee-affb-3fe2558b7927'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:10.624380 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b9285019-d124-45ee-affb-3fe2558b7927 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e95ad0e5c9946e0bc767bbfd85009bc', 'request_id': 'req-b9285019-d124-45ee-affb-3fe2558b7927'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:10.636881 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b9285019-d124-45ee-affb-3fe2558b7927 admin admin] The token cache is being invalidate because project 2e95ad0e5c9946e0bc767bbfd85009bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:23:10.637111 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b9285019-d124-45ee-affb-3fe2558b7927 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:10.637747 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b9285019-d124-45ee-affb-3fe2558b7927 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:10.639224 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1421/2845] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:23:10 2022] DELETE /identity/v3/projects/2e95ad0e5c9946e0bc767bbfd85009bc => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:23:10.643306 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fab823-8d8f-4cd3-8f5e-854cb89c8cbc service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:10.653772 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fab823-8d8f-4cd3-8f5e-854cb89c8cbc service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:10.710396 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c1f0c0-60ec-48d1-b349-b8384c61bc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:10.711028 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1f0c0-60ec-48d1-b349-b8384c61bc6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:10.711306 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1f0c0-60ec-48d1-b349-b8384c61bc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:10.711589 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1f0c0-60ec-48d1-b349-b8384c61bc6d admin admin] PATH_INFO: `/v3/users/435bfca952554cbf829bd54e07ba7d13` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:10.713399 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c1f0c0-60ec-48d1-b349-b8384c61bc6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=435bfca952554cbf829bd54e07ba7d13)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:10.715301 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c1f0c0-60ec-48d1-b349-b8384c61bc6d admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:10.878182 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e3c1f0c0-60ec-48d1-b349-b8384c61bc6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '435bfca952554cbf829bd54e07ba7d13'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:10.878828 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e3c1f0c0-60ec-48d1-b349-b8384c61bc6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '435bfca952554cbf829bd54e07ba7d13'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:10.933142 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e3c1f0c0-60ec-48d1-b349-b8384c61bc6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '435bfca952554cbf829bd54e07ba7d13'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:10.933685 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e3c1f0c0-60ec-48d1-b349-b8384c61bc6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '435bfca952554cbf829bd54e07ba7d13'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:10.945703 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-e3c1f0c0-60ec-48d1-b349-b8384c61bc6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '435bfca952554cbf829bd54e07ba7d13'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:10.958523 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1425/2846] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:23:10 2022] DELETE /identity/v3/users/435bfca952554cbf829bd54e07ba7d13 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:23:11.369131 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9285019-d124-45ee-affb-3fe2558b7927 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:11.372085 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b9285019-d124-45ee-affb-3fe2558b7927 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:11.386725 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9285019-d124-45ee-affb-3fe2558b7927 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:11.446688 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73836d98-9999-4bba-87c1-b74de838fafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:11.447368 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73836d98-9999-4bba-87c1-b74de838fafd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:11.447649 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73836d98-9999-4bba-87c1-b74de838fafd admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:11.447927 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73836d98-9999-4bba-87c1-b74de838fafd admin admin] PATH_INFO: `/v3/projects/1ce281cf7fb84d0e834aed98d0b7c4b9` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:11.449061 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73836d98-9999-4bba-87c1-b74de838fafd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ce281cf7fb84d0e834aed98d0b7c4b9)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:11.450958 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73836d98-9999-4bba-87c1-b74de838fafd admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:11.548566 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-73836d98-9999-4bba-87c1-b74de838fafd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ce281cf7fb84d0e834aed98d0b7c4b9', 'request_id': 'req-73836d98-9999-4bba-87c1-b74de838fafd'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:11.633517 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-73836d98-9999-4bba-87c1-b74de838fafd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ce281cf7fb84d0e834aed98d0b7c4b9', 'request_id': 'req-73836d98-9999-4bba-87c1-b74de838fafd'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:11.639486 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-73836d98-9999-4bba-87c1-b74de838fafd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ce281cf7fb84d0e834aed98d0b7c4b9', 'request_id': 'req-73836d98-9999-4bba-87c1-b74de838fafd'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:11.651275 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-73836d98-9999-4bba-87c1-b74de838fafd admin admin] The token cache is being invalidate because project 1ce281cf7fb84d0e834aed98d0b7c4b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:23:11.651565 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-73836d98-9999-4bba-87c1-b74de838fafd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:11.652207 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-73836d98-9999-4bba-87c1-b74de838fafd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:11.653715 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1422/2847] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:23:11 2022] DELETE /identity/v3/projects/1ce281cf7fb84d0e834aed98d0b7c4b9 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:23:11.658623 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375eca79-d3b6-4097-ad10-16b101fcf9d7 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:11.659104 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375eca79-d3b6-4097-ad10-16b101fcf9d7 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:11.659225 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375eca79-d3b6-4097-ad10-16b101fcf9d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:11.679131 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-375eca79-d3b6-4097-ad10-16b101fcf9d7 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:23:11.680332 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-375eca79-d3b6-4097-ad10-16b101fcf9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:11.738104 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1426/2848] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:23:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:23:11.746017 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f77f4-cc55-4f6e-bceb-36424d7971bd None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:11.746302 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f77f4-cc55-4f6e-bceb-36424d7971bd None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:11.746577 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f77f4-cc55-4f6e-bceb-36424d7971bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:11.766138 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-bc3f77f4-cc55-4f6e-bceb-36424d7971bd None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:23:11.767412 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bc3f77f4-cc55-4f6e-bceb-36424d7971bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:11.775867 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1423/2849] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:23:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:23:11.780211 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375eca79-d3b6-4097-ad10-16b101fcf9d7 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:11.786357 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:23:11.790147 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375eca79-d3b6-4097-ad10-16b101fcf9d7 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:11.798429 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79edc18-b3bb-4514-95d4-c0723c62e9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:11.798429 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79edc18-b3bb-4514-95d4-c0723c62e9ce admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:11.798429 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79edc18-b3bb-4514-95d4-c0723c62e9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:11.798429 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79edc18-b3bb-4514-95d4-c0723c62e9ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:11.798898 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79edc18-b3bb-4514-95d4-c0723c62e9ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:23:11.798898 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79edc18-b3bb-4514-95d4-c0723c62e9ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:11.799220 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79edc18-b3bb-4514-95d4-c0723c62e9ce admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:11.810877 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1427/2850] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:23:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:23:11.815108 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f77f4-cc55-4f6e-bceb-36424d7971bd None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:11.824273 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f77f4-cc55-4f6e-bceb-36424d7971bd None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:11.828903 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9113d8aa-24de-4dec-9c1b-ab63a3ac950f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:11.829492 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9113d8aa-24de-4dec-9c1b-ab63a3ac950f admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:11.829770 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9113d8aa-24de-4dec-9c1b-ab63a3ac950f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:11.830040 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9113d8aa-24de-4dec-9c1b-ab63a3ac950f admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:11.830661 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9113d8aa-24de-4dec-9c1b-ab63a3ac950f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:11.832551 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9113d8aa-24de-4dec-9c1b-ab63a3ac950f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:11.931888 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1424/2851] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:23:11 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:23:11.936040 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79edc18-b3bb-4514-95d4-c0723c62e9ce admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:11.946094 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79edc18-b3bb-4514-95d4-c0723c62e9ce admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:12.002027 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b6d6da-0c36-4710-a9b6-4ffc75794f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:12.002627 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6d6da-0c36-4710-a9b6-4ffc75794f06 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:12.002893 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6d6da-0c36-4710-a9b6-4ffc75794f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:12.003162 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6d6da-0c36-4710-a9b6-4ffc75794f06 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:12.003788 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b6d6da-0c36-4710-a9b6-4ffc75794f06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:12.005988 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b6d6da-0c36-4710-a9b6-4ffc75794f06 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:12.090914 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1428/2852] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:23:11 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:23:12.094915 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9113d8aa-24de-4dec-9c1b-ab63a3ac950f admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:12.105354 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9113d8aa-24de-4dec-9c1b-ab63a3ac950f admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:12.109891 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3be851-8d39-49cd-a589-8f5e27c944c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:12.110489 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3be851-8d39-49cd-a589-8f5e27c944c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:12.110766 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3be851-8d39-49cd-a589-8f5e27c944c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:12.111036 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3be851-8d39-49cd-a589-8f5e27c944c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:12.111661 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3be851-8d39-49cd-a589-8f5e27c944c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:12.113277 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3be851-8d39-49cd-a589-8f5e27c944c6 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:12.123016 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1425/2853] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:23:12.127026 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b6d6da-0c36-4710-a9b6-4ffc75794f06 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:12.138164 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b6d6da-0c36-4710-a9b6-4ffc75794f06 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:12.142896 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358830b5-4638-4562-b588-f73deda36060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:12.143490 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358830b5-4638-4562-b588-f73deda36060 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:12.143775 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358830b5-4638-4562-b588-f73deda36060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:12.144053 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358830b5-4638-4562-b588-f73deda36060 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:12.144733 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358830b5-4638-4562-b588-f73deda36060 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:12.146313 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358830b5-4638-4562-b588-f73deda36060 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:12.157562 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1429/2854] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:23:12.161559 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3be851-8d39-49cd-a589-8f5e27c944c6 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:12.171494 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3be851-8d39-49cd-a589-8f5e27c944c6 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:12.176057 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635bbb90-6bfe-410e-9403-80013d53be9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:12.176726 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635bbb90-6bfe-410e-9403-80013d53be9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:12.177013 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635bbb90-6bfe-410e-9403-80013d53be9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:12.177292 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635bbb90-6bfe-410e-9403-80013d53be9f admin admin] PATH_INFO: `/v3/projects/714793a461a440d68094a2f186f9dc9b/users/7f7d9c06f8b540098aabe491123e60be/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:12.191480 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635bbb90-6bfe-410e-9403-80013d53be9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=714793a461a440d68094a2f186f9dc9b, user_id=7f7d9c06f8b540098aabe491123e60be, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:12.194164 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635bbb90-6bfe-410e-9403-80013d53be9f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:12.267965 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1426/2855] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:23:12 2022] PUT /identity/v3/projects/714793a461a440d68094a2f186f9dc9b/users/7f7d9c06f8b540098aabe491123e60be/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:23:12.278221 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd60e0d-1c2f-437a-960e-2a2173081ccd None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:12.281485 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd60e0d-1c2f-437a-960e-2a2173081ccd None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:12.281485 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd60e0d-1c2f-437a-960e-2a2173081ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:12.306639 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-2dd60e0d-1c2f-437a-960e-2a2173081ccd None None] MFA Rules not processed for user `7f7d9c06f8b540098aabe491123e60be`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:23:12.307867 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-2dd60e0d-1c2f-437a-960e-2a2173081ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:12.371513 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1430/2856] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:23:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:23:12.375916 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635bbb90-6bfe-410e-9403-80013d53be9f admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:12.381995 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:INSERT=4,keystone:DELETE=10,keystone:SELECT=86 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:23:12.439008 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635bbb90-6bfe-410e-9403-80013d53be9f admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:12.444280 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c4006-0cfe-4177-a9bd-3fc527048cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:12.444912 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c4006-0cfe-4177-a9bd-3fc527048cea admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:12.445198 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c4006-0cfe-4177-a9bd-3fc527048cea admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:12.445581 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c4006-0cfe-4177-a9bd-3fc527048cea admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:12.446227 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c4006-0cfe-4177-a9bd-3fc527048cea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:12.449234 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c4006-0cfe-4177-a9bd-3fc527048cea admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:12.560880 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1427/2857] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:23:12 2022] POST /identity/v3/projects => generated 379 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:23:12.564951 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd60e0d-1c2f-437a-960e-2a2173081ccd None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:12.576361 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd60e0d-1c2f-437a-960e-2a2173081ccd None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:12.633309 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb81fbf-3593-4f25-9e72-10482c3ead0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:12.633840 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb81fbf-3593-4f25-9e72-10482c3ead0a admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:12.634986 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb81fbf-3593-4f25-9e72-10482c3ead0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:12.634986 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb81fbf-3593-4f25-9e72-10482c3ead0a admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:12.635632 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb81fbf-3593-4f25-9e72-10482c3ead0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:12.637595 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb81fbf-3593-4f25-9e72-10482c3ead0a admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:12.645626 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:23:12.707500 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1431/2858] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:23:12 2022] POST /identity/v3/users => generated 290 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:23:12.711423 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c4006-0cfe-4177-a9bd-3fc527048cea admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:12.721972 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c4006-0cfe-4177-a9bd-3fc527048cea admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:12.727090 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38603183-9b99-49cd-bec3-68cf38acaa5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:12.728280 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38603183-9b99-49cd-bec3-68cf38acaa5a admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:12.728599 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38603183-9b99-49cd-bec3-68cf38acaa5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:12.733564 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38603183-9b99-49cd-bec3-68cf38acaa5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:12.733564 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38603183-9b99-49cd-bec3-68cf38acaa5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:12.733564 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38603183-9b99-49cd-bec3-68cf38acaa5a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:12.742127 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1428/2859] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:23:12.746716 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb81fbf-3593-4f25-9e72-10482c3ead0a admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:12.757163 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb81fbf-3593-4f25-9e72-10482c3ead0a admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:12.762048 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda0c4f-4943-4f76-9e08-56076739e099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:12.768008 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda0c4f-4943-4f76-9e08-56076739e099 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:12.768008 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda0c4f-4943-4f76-9e08-56076739e099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:12.768008 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda0c4f-4943-4f76-9e08-56076739e099 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:12.768008 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda0c4f-4943-4f76-9e08-56076739e099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:12.768008 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda0c4f-4943-4f76-9e08-56076739e099 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:12.775752 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1432/2860] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:23:12.779967 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38603183-9b99-49cd-bec3-68cf38acaa5a admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:12.795062 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38603183-9b99-49cd-bec3-68cf38acaa5a admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:12.804975 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418c64bc-3b07-4162-962f-9d87db670f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:12.804975 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c64bc-3b07-4162-962f-9d87db670f19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:12.804975 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c64bc-3b07-4162-962f-9d87db670f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:12.804975 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c64bc-3b07-4162-962f-9d87db670f19 admin admin] PATH_INFO: `/v3/projects/7c5361038f4640ccb0b532f892bf4ce1/users/e4090aed0f3e4f848544e2e3b77c8cb0/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:12.816527 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418c64bc-3b07-4162-962f-9d87db670f19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c5361038f4640ccb0b532f892bf4ce1, user_id=e4090aed0f3e4f848544e2e3b77c8cb0, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:12.819181 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418c64bc-3b07-4162-962f-9d87db670f19 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:12.916227 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1429/2861] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:23:12 2022] PUT /identity/v3/projects/7c5361038f4640ccb0b532f892bf4ce1/users/e4090aed0f3e4f848544e2e3b77c8cb0/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:23:12.927630 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc171ad2-dde0-4b01-9201-9f89aa7249a0 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:12.927916 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc171ad2-dde0-4b01-9201-9f89aa7249a0 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:12.928188 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc171ad2-dde0-4b01-9201-9f89aa7249a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:12.957591 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-fc171ad2-dde0-4b01-9201-9f89aa7249a0 None None] MFA Rules not processed for user `e4090aed0f3e4f848544e2e3b77c8cb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:23:12.958826 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-fc171ad2-dde0-4b01-9201-9f89aa7249a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:13.022317 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1433/2862] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:23:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:23:13.027426 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418c64bc-3b07-4162-962f-9d87db670f19 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:13.087915 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418c64bc-3b07-4162-962f-9d87db670f19 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:13.092500 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69531d87-5add-455d-8b49-ea11ae22c185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:13.093092 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69531d87-5add-455d-8b49-ea11ae22c185 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:13.093371 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69531d87-5add-455d-8b49-ea11ae22c185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:13.093641 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69531d87-5add-455d-8b49-ea11ae22c185 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:13.094271 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69531d87-5add-455d-8b49-ea11ae22c185 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:13.096142 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69531d87-5add-455d-8b49-ea11ae22c185 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:13.179550 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1430/2863] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:23:13 2022] POST /identity/v3/projects => generated 377 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:23:13.183902 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc171ad2-dde0-4b01-9201-9f89aa7249a0 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:13.194368 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc171ad2-dde0-4b01-9201-9f89aa7249a0 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:13.252927 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431354a-52b1-448f-912f-308bb0f0dd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:13.253528 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431354a-52b1-448f-912f-308bb0f0dd13 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:13.253807 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431354a-52b1-448f-912f-308bb0f0dd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:13.254081 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431354a-52b1-448f-912f-308bb0f0dd13 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:13.254963 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4431354a-52b1-448f-912f-308bb0f0dd13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:13.256912 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4431354a-52b1-448f-912f-308bb0f0dd13 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:13.367009 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1434/2864] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:23:13 2022] POST /identity/v3/users => generated 295 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:23:13.371530 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69531d87-5add-455d-8b49-ea11ae22c185 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:13.382265 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69531d87-5add-455d-8b49-ea11ae22c185 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:13.387995 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452cbbb8-baa8-41d0-b662-a9169f9c6f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:13.388698 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452cbbb8-baa8-41d0-b662-a9169f9c6f4f admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:13.388999 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452cbbb8-baa8-41d0-b662-a9169f9c6f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:13.389270 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452cbbb8-baa8-41d0-b662-a9169f9c6f4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:13.389902 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452cbbb8-baa8-41d0-b662-a9169f9c6f4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:13.391477 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452cbbb8-baa8-41d0-b662-a9169f9c6f4f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:13.402950 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1431/2865] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:23:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:23:13.407041 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431354a-52b1-448f-912f-308bb0f0dd13 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:13.417108 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431354a-52b1-448f-912f-308bb0f0dd13 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:13.421736 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba1bcf-1987-4cc2-a69d-ab2f424a5e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:13.422384 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba1bcf-1987-4cc2-a69d-ab2f424a5e29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:13.422672 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba1bcf-1987-4cc2-a69d-ab2f424a5e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:13.422946 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba1bcf-1987-4cc2-a69d-ab2f424a5e29 admin admin] PATH_INFO: `/v3/projects/c60292222f684500bb0cae4617bbeed3/users/c9bb101baeae4a958826dc032f155114/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:13.438085 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbba1bcf-1987-4cc2-a69d-ab2f424a5e29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c60292222f684500bb0cae4617bbeed3, user_id=c9bb101baeae4a958826dc032f155114, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:13.440762 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbba1bcf-1987-4cc2-a69d-ab2f424a5e29 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:13.521590 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1435/2866] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:23:13 2022] PUT /identity/v3/projects/c60292222f684500bb0cae4617bbeed3/users/c9bb101baeae4a958826dc032f155114/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:23:13.526219 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452cbbb8-baa8-41d0-b662-a9169f9c6f4f admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:13.535934 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452cbbb8-baa8-41d0-b662-a9169f9c6f4f admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:13.561961 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:23:13.592298 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8a959-5df5-40ab-9d60-89f98f127003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:13.592942 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8a959-5df5-40ab-9d60-89f98f127003 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:13.593228 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8a959-5df5-40ab-9d60-89f98f127003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:13.593505 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8a959-5df5-40ab-9d60-89f98f127003 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:13.594166 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc8a959-5df5-40ab-9d60-89f98f127003 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:13.596094 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc8a959-5df5-40ab-9d60-89f98f127003 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:13.606252 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1432/2867] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:23:13 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:23:13.610369 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba1bcf-1987-4cc2-a69d-ab2f424a5e29 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:13.620685 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba1bcf-1987-4cc2-a69d-ab2f424a5e29 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:13.625306 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6657f45d-6b47-4f05-b623-a29b410d2612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:13.625951 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657f45d-6b47-4f05-b623-a29b410d2612 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:13.626282 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657f45d-6b47-4f05-b623-a29b410d2612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:13.626563 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657f45d-6b47-4f05-b623-a29b410d2612 admin admin] PATH_INFO: `/v3/projects/c60292222f684500bb0cae4617bbeed3/users/c9bb101baeae4a958826dc032f155114/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:13.628484 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6657f45d-6b47-4f05-b623-a29b410d2612 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c60292222f684500bb0cae4617bbeed3, user_id=c9bb101baeae4a958826dc032f155114, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:13.631391 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6657f45d-6b47-4f05-b623-a29b410d2612 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:13.642150 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1436/2868] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:23:13 2022] PUT /identity/v3/projects/c60292222f684500bb0cae4617bbeed3/users/c9bb101baeae4a958826dc032f155114/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:23:13.652750 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795a2d9-4c1b-47b8-b87f-cbfbb04e4734 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:13.653045 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795a2d9-4c1b-47b8-b87f-cbfbb04e4734 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:13.653327 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795a2d9-4c1b-47b8-b87f-cbfbb04e4734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:13.682039 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-7795a2d9-4c1b-47b8-b87f-cbfbb04e4734 None None] MFA Rules not processed for user `c9bb101baeae4a958826dc032f155114`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:23:13.683262 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7795a2d9-4c1b-47b8-b87f-cbfbb04e4734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:13.751896 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1433/2869] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:23:13 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:23:13.764735 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6657f45d-6b47-4f05-b623-a29b410d2612 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:13.766455 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-6657f45d-6b47-4f05-b623-a29b410d2612 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:13.788597 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:23:13.824118 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6657f45d-6b47-4f05-b623-a29b410d2612 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:13.827736 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df5381b-b903-4bbe-80c8-eb1a64282211 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:13.828372 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5381b-b903-4bbe-80c8-eb1a64282211 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:13.828734 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5381b-b903-4bbe-80c8-eb1a64282211 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:13.829009 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5381b-b903-4bbe-80c8-eb1a64282211 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:13.837835 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df5381b-b903-4bbe-80c8-eb1a64282211 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:13.839255 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df5381b-b903-4bbe-80c8-eb1a64282211 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:13.848644 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1437/2870] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:23:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:23:13.969649 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7795a2d9-4c1b-47b8-b87f-cbfbb04e4734 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:13.971101 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7795a2d9-4c1b-47b8-b87f-cbfbb04e4734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:13.983386 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7795a2d9-4c1b-47b8-b87f-cbfbb04e4734 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:13.986909 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5272cf2-3233-4360-9f6c-663b0376eca3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:13.987511 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5272cf2-3233-4360-9f6c-663b0376eca3 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:13.987791 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5272cf2-3233-4360-9f6c-663b0376eca3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:13.988063 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5272cf2-3233-4360-9f6c-663b0376eca3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:14.044421 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5272cf2-3233-4360-9f6c-663b0376eca3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:14.045868 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5272cf2-3233-4360-9f6c-663b0376eca3 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:14.054890 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1434/2871] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:23:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:23:17.996438 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df5381b-b903-4bbe-80c8-eb1a64282211 service cinder] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:18.057593 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df5381b-b903-4bbe-80c8-eb1a64282211 service cinder] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:18.062088 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e319383-8310-4568-9739-91442886653d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:18.062721 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e319383-8310-4568-9739-91442886653d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:18.062998 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e319383-8310-4568-9739-91442886653d admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:18.063269 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e319383-8310-4568-9739-91442886653d admin admin] PATH_INFO: `/v3/users/7f7d9c06f8b540098aabe491123e60be` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:18.064374 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e319383-8310-4568-9739-91442886653d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f7d9c06f8b540098aabe491123e60be)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:18.066306 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e319383-8310-4568-9739-91442886653d admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:18.238060 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-4e319383-8310-4568-9739-91442886653d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f7d9c06f8b540098aabe491123e60be'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:18.238707 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-4e319383-8310-4568-9739-91442886653d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f7d9c06f8b540098aabe491123e60be'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:18.303850 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-4e319383-8310-4568-9739-91442886653d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f7d9c06f8b540098aabe491123e60be'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:18.304491 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-4e319383-8310-4568-9739-91442886653d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f7d9c06f8b540098aabe491123e60be'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:18.317100 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-4e319383-8310-4568-9739-91442886653d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f7d9c06f8b540098aabe491123e60be'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:18.329671 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1438/2872] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:23:17 2022] DELETE /identity/v3/users/7f7d9c06f8b540098aabe491123e60be => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:23:18.334125 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa0f51-6d3e-4c7d-959a-a8e79286a52c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:18.334607 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa0f51-6d3e-4c7d-959a-a8e79286a52c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:18.334709 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa0f51-6d3e-4c7d-959a-a8e79286a52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:18.353521 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-d4aa0f51-6d3e-4c7d-959a-a8e79286a52c None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:23:18.354755 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d4aa0f51-6d3e-4c7d-959a-a8e79286a52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:18.397739 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:23:18.412588 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1435/2873] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:23:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:23:18.422423 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e319383-8310-4568-9739-91442886653d admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:18.423822 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-4e319383-8310-4568-9739-91442886653d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:18.472278 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:23:18.483861 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e319383-8310-4568-9739-91442886653d admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:18.487545 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5cd596-de2f-446e-be5d-5fec1fe5088d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:18.488153 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cd596-de2f-446e-be5d-5fec1fe5088d service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:18.488437 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cd596-de2f-446e-be5d-5fec1fe5088d service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:18.488741 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cd596-de2f-446e-be5d-5fec1fe5088d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:18.498379 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5cd596-de2f-446e-be5d-5fec1fe5088d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:18.499823 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5cd596-de2f-446e-be5d-5fec1fe5088d service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:18.509419 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1439/2874] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:23:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:23:18.917403 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa0f51-6d3e-4c7d-959a-a8e79286a52c None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:18.918995 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d4aa0f51-6d3e-4c7d-959a-a8e79286a52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:18.934012 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa0f51-6d3e-4c7d-959a-a8e79286a52c None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:18.945548 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ecaea-bcd3-47c3-970b-31091e29ceef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:18.946200 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ecaea-bcd3-47c3-970b-31091e29ceef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:18.946484 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ecaea-bcd3-47c3-970b-31091e29ceef admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:18.946757 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ecaea-bcd3-47c3-970b-31091e29ceef admin admin] PATH_INFO: `/v3/projects/714793a461a440d68094a2f186f9dc9b` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:18.947918 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ecaea-bcd3-47c3-970b-31091e29ceef admin admin] RBAC: Authorizing `identity:delete_project(project_id=714793a461a440d68094a2f186f9dc9b)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:18.950578 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ecaea-bcd3-47c3-970b-31091e29ceef admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:19.077213 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f95ecaea-bcd3-47c3-970b-31091e29ceef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '714793a461a440d68094a2f186f9dc9b', 'request_id': 'req-f95ecaea-bcd3-47c3-970b-31091e29ceef'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:19.138100 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f95ecaea-bcd3-47c3-970b-31091e29ceef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '714793a461a440d68094a2f186f9dc9b', 'request_id': 'req-f95ecaea-bcd3-47c3-970b-31091e29ceef'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:19.143959 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f95ecaea-bcd3-47c3-970b-31091e29ceef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '714793a461a440d68094a2f186f9dc9b', 'request_id': 'req-f95ecaea-bcd3-47c3-970b-31091e29ceef'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:19.155736 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f95ecaea-bcd3-47c3-970b-31091e29ceef admin admin] The token cache is being invalidate because project 714793a461a440d68094a2f186f9dc9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:23:19.156026 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f95ecaea-bcd3-47c3-970b-31091e29ceef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:19.156697 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f95ecaea-bcd3-47c3-970b-31091e29ceef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:19.158186 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1436/2875] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:23:18 2022] DELETE /identity/v3/projects/714793a461a440d68094a2f186f9dc9b => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:23:19.162140 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5cd596-de2f-446e-be5d-5fec1fe5088d service neutron] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:19.172187 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5cd596-de2f-446e-be5d-5fec1fe5088d service neutron] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:19.230468 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d2076c-66e0-4ebd-83e4-6b4b966f9bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:19.231565 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2076c-66e0-4ebd-83e4-6b4b966f9bd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:19.231565 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2076c-66e0-4ebd-83e4-6b4b966f9bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:19.231797 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2076c-66e0-4ebd-83e4-6b4b966f9bd2 admin admin] PATH_INFO: `/v3/users/e4090aed0f3e4f848544e2e3b77c8cb0` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:19.232751 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d2076c-66e0-4ebd-83e4-6b4b966f9bd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4090aed0f3e4f848544e2e3b77c8cb0)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:19.234664 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d2076c-66e0-4ebd-83e4-6b4b966f9bd2 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:19.412252 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-33d2076c-66e0-4ebd-83e4-6b4b966f9bd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4090aed0f3e4f848544e2e3b77c8cb0'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:19.412841 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-33d2076c-66e0-4ebd-83e4-6b4b966f9bd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4090aed0f3e4f848544e2e3b77c8cb0'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:19.484564 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-33d2076c-66e0-4ebd-83e4-6b4b966f9bd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4090aed0f3e4f848544e2e3b77c8cb0'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:19.485198 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-33d2076c-66e0-4ebd-83e4-6b4b966f9bd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4090aed0f3e4f848544e2e3b77c8cb0'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:19.497403 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-33d2076c-66e0-4ebd-83e4-6b4b966f9bd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4090aed0f3e4f848544e2e3b77c8cb0'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:19.509673 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1440/2876] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:23:19 2022] DELETE /identity/v3/users/e4090aed0f3e4f848544e2e3b77c8cb0 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:23:19.892482 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ecaea-bcd3-47c3-970b-31091e29ceef admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:19.893948 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f95ecaea-bcd3-47c3-970b-31091e29ceef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:19.907978 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ecaea-bcd3-47c3-970b-31091e29ceef admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:19.969305 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae57c49e-5964-494a-8255-5860d3f84dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:19.969860 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae57c49e-5964-494a-8255-5860d3f84dda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:19.970145 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae57c49e-5964-494a-8255-5860d3f84dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:19.970424 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae57c49e-5964-494a-8255-5860d3f84dda admin admin] PATH_INFO: `/v3/projects/7c5361038f4640ccb0b532f892bf4ce1` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:19.971551 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae57c49e-5964-494a-8255-5860d3f84dda admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c5361038f4640ccb0b532f892bf4ce1)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:19.973497 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae57c49e-5964-494a-8255-5860d3f84dda admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:20.073694 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-ae57c49e-5964-494a-8255-5860d3f84dda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c5361038f4640ccb0b532f892bf4ce1', 'request_id': 'req-ae57c49e-5964-494a-8255-5860d3f84dda'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:20.136179 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-ae57c49e-5964-494a-8255-5860d3f84dda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c5361038f4640ccb0b532f892bf4ce1', 'request_id': 'req-ae57c49e-5964-494a-8255-5860d3f84dda'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:20.141977 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-ae57c49e-5964-494a-8255-5860d3f84dda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c5361038f4640ccb0b532f892bf4ce1', 'request_id': 'req-ae57c49e-5964-494a-8255-5860d3f84dda'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:20.153159 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-ae57c49e-5964-494a-8255-5860d3f84dda admin admin] The token cache is being invalidate because project 7c5361038f4640ccb0b532f892bf4ce1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:23:20.153451 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-ae57c49e-5964-494a-8255-5860d3f84dda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:20.154165 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-ae57c49e-5964-494a-8255-5860d3f84dda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:20.155660 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1437/2877] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:23:19 2022] DELETE /identity/v3/projects/7c5361038f4640ccb0b532f892bf4ce1 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:23:20.159721 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d2076c-66e0-4ebd-83e4-6b4b966f9bd2 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:20.161134 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-33d2076c-66e0-4ebd-83e4-6b4b966f9bd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:20.174659 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d2076c-66e0-4ebd-83e4-6b4b966f9bd2 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:20.231296 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64e80e-14de-4c40-aecd-99768728a06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:20.231881 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64e80e-14de-4c40-aecd-99768728a06c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:20.232164 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64e80e-14de-4c40-aecd-99768728a06c admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:20.232826 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64e80e-14de-4c40-aecd-99768728a06c admin admin] PATH_INFO: `/v3/users/c9bb101baeae4a958826dc032f155114` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:20.233615 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e64e80e-14de-4c40-aecd-99768728a06c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9bb101baeae4a958826dc032f155114)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:20.235533 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e64e80e-14de-4c40-aecd-99768728a06c admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:20.240320 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:23:20.451808 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2e64e80e-14de-4c40-aecd-99768728a06c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9bb101baeae4a958826dc032f155114'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:20.452452 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2e64e80e-14de-4c40-aecd-99768728a06c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9bb101baeae4a958826dc032f155114'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:20.518532 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2e64e80e-14de-4c40-aecd-99768728a06c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9bb101baeae4a958826dc032f155114'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:20.519136 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2e64e80e-14de-4c40-aecd-99768728a06c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9bb101baeae4a958826dc032f155114'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:20.531596 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-2e64e80e-14de-4c40-aecd-99768728a06c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9bb101baeae4a958826dc032f155114'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:20.544086 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1441/2878] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:23:20 2022] DELETE /identity/v3/users/c9bb101baeae4a958826dc032f155114 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:23:20.951435 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae57c49e-5964-494a-8255-5860d3f84dda admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:20.952925 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ae57c49e-5964-494a-8255-5860d3f84dda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:20.964780 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:23:20.968916 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae57c49e-5964-494a-8255-5860d3f84dda admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:21.028535 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934a5c2-04a8-443a-bfd5-fd68f5128efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:21.029169 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934a5c2-04a8-443a-bfd5-fd68f5128efc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:21.029450 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934a5c2-04a8-443a-bfd5-fd68f5128efc admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:21.029724 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934a5c2-04a8-443a-bfd5-fd68f5128efc admin admin] PATH_INFO: `/v3/projects/c60292222f684500bb0cae4617bbeed3` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:21.030894 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934a5c2-04a8-443a-bfd5-fd68f5128efc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c60292222f684500bb0cae4617bbeed3)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:21.032835 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934a5c2-04a8-443a-bfd5-fd68f5128efc admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:21.131304 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6934a5c2-04a8-443a-bfd5-fd68f5128efc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c60292222f684500bb0cae4617bbeed3', 'request_id': 'req-6934a5c2-04a8-443a-bfd5-fd68f5128efc'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:21.210476 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6934a5c2-04a8-443a-bfd5-fd68f5128efc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c60292222f684500bb0cae4617bbeed3', 'request_id': 'req-6934a5c2-04a8-443a-bfd5-fd68f5128efc'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:21.217492 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6934a5c2-04a8-443a-bfd5-fd68f5128efc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c60292222f684500bb0cae4617bbeed3', 'request_id': 'req-6934a5c2-04a8-443a-bfd5-fd68f5128efc'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:21.229087 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6934a5c2-04a8-443a-bfd5-fd68f5128efc admin admin] The token cache is being invalidate because project c60292222f684500bb0cae4617bbeed3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:23:21.229962 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6934a5c2-04a8-443a-bfd5-fd68f5128efc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:21.231118 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6934a5c2-04a8-443a-bfd5-fd68f5128efc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:21.232624 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1438/2879] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:23:20 2022] DELETE /identity/v3/projects/c60292222f684500bb0cae4617bbeed3 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:23:21.238724 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b5aaf-1d8c-48a2-b6e9-ed57d3636ab4 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:21.239208 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b5aaf-1d8c-48a2-b6e9-ed57d3636ab4 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:21.239324 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b5aaf-1d8c-48a2-b6e9-ed57d3636ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:21.259259 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-5a6b5aaf-1d8c-48a2-b6e9-ed57d3636ab4 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:23:21.260664 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5a6b5aaf-1d8c-48a2-b6e9-ed57d3636ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:21.319504 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1442/2880] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:23:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:23:21.328307 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05076cfe-4812-4350-b750-078c43bda886 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:21.328667 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05076cfe-4812-4350-b750-078c43bda886 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:21.328973 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05076cfe-4812-4350-b750-078c43bda886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:21.349518 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-05076cfe-4812-4350-b750-078c43bda886 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:23:21.350720 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-05076cfe-4812-4350-b750-078c43bda886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:21.359771 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1439/2881] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:23:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:23:21.363989 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b5aaf-1d8c-48a2-b6e9-ed57d3636ab4 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:21.374218 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b5aaf-1d8c-48a2-b6e9-ed57d3636ab4 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:21.378809 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a841bda-a62e-48fa-bf40-9618517241f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:21.379405 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a841bda-a62e-48fa-bf40-9618517241f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:21.379947 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a841bda-a62e-48fa-bf40-9618517241f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:21.380244 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a841bda-a62e-48fa-bf40-9618517241f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:21.380899 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a841bda-a62e-48fa-bf40-9618517241f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:23:21.381204 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a841bda-a62e-48fa-bf40-9618517241f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:21.382900 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a841bda-a62e-48fa-bf40-9618517241f6 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:21.394563 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1443/2882] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:23:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:23:21.398918 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05076cfe-4812-4350-b750-078c43bda886 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:21.409060 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05076cfe-4812-4350-b750-078c43bda886 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:21.413669 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f51bd-d202-495a-be25-c7f399fb83ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:21.414259 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f51bd-d202-495a-be25-c7f399fb83ee admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:21.414540 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f51bd-d202-495a-be25-c7f399fb83ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:21.414816 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f51bd-d202-495a-be25-c7f399fb83ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:21.415444 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f51bd-d202-495a-be25-c7f399fb83ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:21.418452 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f51bd-d202-495a-be25-c7f399fb83ee admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:21.503493 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1440/2883] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:23:21 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:23:21.507610 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a841bda-a62e-48fa-bf40-9618517241f6 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:21.517976 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a841bda-a62e-48fa-bf40-9618517241f6 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:21.574914 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f1e8b-aab6-4e22-b382-07d8b8030b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:21.575760 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f1e8b-aab6-4e22-b382-07d8b8030b0e admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:21.576048 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f1e8b-aab6-4e22-b382-07d8b8030b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:21.576323 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f1e8b-aab6-4e22-b382-07d8b8030b0e admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:21.576980 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f1e8b-aab6-4e22-b382-07d8b8030b0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:21.578890 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f1e8b-aab6-4e22-b382-07d8b8030b0e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:21.674437 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:23:21.679397 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1444/2884] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:23:21 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:23:21.683565 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f51bd-d202-495a-be25-c7f399fb83ee admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:21.694030 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f51bd-d202-495a-be25-c7f399fb83ee admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:21.699616 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455bd5a-97bc-4d39-88b5-9bca6dc32382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:21.700212 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455bd5a-97bc-4d39-88b5-9bca6dc32382 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:21.700559 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455bd5a-97bc-4d39-88b5-9bca6dc32382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:21.700853 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455bd5a-97bc-4d39-88b5-9bca6dc32382 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:21.701496 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2455bd5a-97bc-4d39-88b5-9bca6dc32382 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:21.703104 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2455bd5a-97bc-4d39-88b5-9bca6dc32382 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:21.713021 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1441/2885] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:23:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:23:21.717778 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f1e8b-aab6-4e22-b382-07d8b8030b0e admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:21.727751 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f1e8b-aab6-4e22-b382-07d8b8030b0e admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:21.733388 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61be5619-c809-4305-a78a-4e8bfe584360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:21.733989 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be5619-c809-4305-a78a-4e8bfe584360 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:21.734272 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be5619-c809-4305-a78a-4e8bfe584360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:21.734553 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be5619-c809-4305-a78a-4e8bfe584360 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:21.735195 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61be5619-c809-4305-a78a-4e8bfe584360 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:21.736955 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61be5619-c809-4305-a78a-4e8bfe584360 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:21.747542 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1445/2886] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:23:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:23:21.751868 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455bd5a-97bc-4d39-88b5-9bca6dc32382 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:21.762779 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455bd5a-97bc-4d39-88b5-9bca6dc32382 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:21.767819 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a291e-b72b-47df-a305-81a447c3ed4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:21.769685 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a291e-b72b-47df-a305-81a447c3ed4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:21.769685 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a291e-b72b-47df-a305-81a447c3ed4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:21.769685 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a291e-b72b-47df-a305-81a447c3ed4a admin admin] PATH_INFO: `/v3/projects/423b284639f6462ba2b44754ccaf8493/users/8d8b46efeb14479d98cd2fabe8dac594/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:21.786338 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a291e-b72b-47df-a305-81a447c3ed4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=423b284639f6462ba2b44754ccaf8493, user_id=8d8b46efeb14479d98cd2fabe8dac594, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:21.789334 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a291e-b72b-47df-a305-81a447c3ed4a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:21.875910 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1442/2887] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:23:21 2022] PUT /identity/v3/projects/423b284639f6462ba2b44754ccaf8493/users/8d8b46efeb14479d98cd2fabe8dac594/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:23:21.946167 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ea742-9b27-4a0a-bd47-461b841ebe68 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:21.946377 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ea742-9b27-4a0a-bd47-461b841ebe68 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:21.946621 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ea742-9b27-4a0a-bd47-461b841ebe68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:21.974288 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-307ea742-9b27-4a0a-bd47-461b841ebe68 None None] MFA Rules not processed for user `8d8b46efeb14479d98cd2fabe8dac594`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:23:21.975502 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-307ea742-9b27-4a0a-bd47-461b841ebe68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:22.037697 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1446/2888] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:23:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:23:22.048934 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a291e-b72b-47df-a305-81a447c3ed4a admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:22.050621 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f86a291e-b72b-47df-a305-81a447c3ed4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:22.083664 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:23:22.108266 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a291e-b72b-47df-a305-81a447c3ed4a admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:22.112121 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd422d-900e-4cbf-978c-855c4c375534 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:22.112947 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd422d-900e-4cbf-978c-855c4c375534 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:22.113398 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd422d-900e-4cbf-978c-855c4c375534 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:22.114063 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd422d-900e-4cbf-978c-855c4c375534 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:22.123077 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cd422d-900e-4cbf-978c-855c4c375534 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:22.124591 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cd422d-900e-4cbf-978c-855c4c375534 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:22.133936 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1443/2889] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:23:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:23:22.153574 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ea742-9b27-4a0a-bd47-461b841ebe68 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:22.215201 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ea742-9b27-4a0a-bd47-461b841ebe68 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:22.219678 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bcc811-2848-433d-a80d-590f9aa01431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:22.220300 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bcc811-2848-433d-a80d-590f9aa01431 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:22.220610 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bcc811-2848-433d-a80d-590f9aa01431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:22.220883 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bcc811-2848-433d-a80d-590f9aa01431 admin admin] PATH_INFO: `/v3/users/8d8b46efeb14479d98cd2fabe8dac594` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:22.222006 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bcc811-2848-433d-a80d-590f9aa01431 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d8b46efeb14479d98cd2fabe8dac594)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:22.223914 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bcc811-2848-433d-a80d-590f9aa01431 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:22.380169 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-60bcc811-2848-433d-a80d-590f9aa01431 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d8b46efeb14479d98cd2fabe8dac594'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:22.380861 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-60bcc811-2848-433d-a80d-590f9aa01431 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d8b46efeb14479d98cd2fabe8dac594'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:22.441010 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-60bcc811-2848-433d-a80d-590f9aa01431 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d8b46efeb14479d98cd2fabe8dac594'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:22.441644 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-60bcc811-2848-433d-a80d-590f9aa01431 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d8b46efeb14479d98cd2fabe8dac594'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:22.453928 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-60bcc811-2848-433d-a80d-590f9aa01431 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d8b46efeb14479d98cd2fabe8dac594'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:22.461559 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:23:22.466683 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1447/2890] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:23:22 2022] DELETE /identity/v3/users/8d8b46efeb14479d98cd2fabe8dac594 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:23:22.471096 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a365c-8960-40bf-8958-be274ec4c4ee None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:22.471375 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a365c-8960-40bf-8958-be274ec4c4ee None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:22.471673 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a365c-8960-40bf-8958-be274ec4c4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:22.490969 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-d62a365c-8960-40bf-8958-be274ec4c4ee None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:23:22.492250 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d62a365c-8960-40bf-8958-be274ec4c4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:22.551724 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1444/2891] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:23:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:23:22.562363 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bcc811-2848-433d-a80d-590f9aa01431 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:22.563764 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-60bcc811-2848-433d-a80d-590f9aa01431 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:22.620629 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bcc811-2848-433d-a80d-590f9aa01431 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:22.624368 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df519e8-fcd2-4df4-bd83-bb40ac64466f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:22.624994 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df519e8-fcd2-4df4-bd83-bb40ac64466f service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:22.625269 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df519e8-fcd2-4df4-bd83-bb40ac64466f service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:22.625541 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df519e8-fcd2-4df4-bd83-bb40ac64466f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:22.634628 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df519e8-fcd2-4df4-bd83-bb40ac64466f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:22.636182 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df519e8-fcd2-4df4-bd83-bb40ac64466f service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:22.645389 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1448/2892] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:23:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:23:23.051663 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a365c-8960-40bf-8958-be274ec4c4ee None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:23.054987 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d62a365c-8960-40bf-8958-be274ec4c4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:23.065438 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a365c-8960-40bf-8958-be274ec4c4ee None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:23.070087 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf51a8-1cb5-42f9-9dd5-f7f883375950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:23.070721 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf51a8-1cb5-42f9-9dd5-f7f883375950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:23.071005 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf51a8-1cb5-42f9-9dd5-f7f883375950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:23.071280 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf51a8-1cb5-42f9-9dd5-f7f883375950 admin admin] PATH_INFO: `/v3/projects/423b284639f6462ba2b44754ccaf8493` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:23.072585 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf51a8-1cb5-42f9-9dd5-f7f883375950 admin admin] RBAC: Authorizing `identity:delete_project(project_id=423b284639f6462ba2b44754ccaf8493)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:23.074992 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf51a8-1cb5-42f9-9dd5-f7f883375950 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:23.176376 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d4cf51a8-1cb5-42f9-9dd5-f7f883375950 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '423b284639f6462ba2b44754ccaf8493', 'request_id': 'req-d4cf51a8-1cb5-42f9-9dd5-f7f883375950'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:23.238797 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d4cf51a8-1cb5-42f9-9dd5-f7f883375950 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '423b284639f6462ba2b44754ccaf8493', 'request_id': 'req-d4cf51a8-1cb5-42f9-9dd5-f7f883375950'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:23.245291 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d4cf51a8-1cb5-42f9-9dd5-f7f883375950 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '423b284639f6462ba2b44754ccaf8493', 'request_id': 'req-d4cf51a8-1cb5-42f9-9dd5-f7f883375950'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:23.256802 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d4cf51a8-1cb5-42f9-9dd5-f7f883375950 admin admin] The token cache is being invalidate because project 423b284639f6462ba2b44754ccaf8493 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:23:23.257097 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d4cf51a8-1cb5-42f9-9dd5-f7f883375950 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:23.257722 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d4cf51a8-1cb5-42f9-9dd5-f7f883375950 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:23.259202 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1445/2893] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:23:23 2022] DELETE /identity/v3/projects/423b284639f6462ba2b44754ccaf8493 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:23:23.264553 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42b721-153f-4a68-99f4-5f6ce0e22e1d None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:23.264843 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42b721-153f-4a68-99f4-5f6ce0e22e1d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:23.265145 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42b721-153f-4a68-99f4-5f6ce0e22e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:23.285230 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-7e42b721-153f-4a68-99f4-5f6ce0e22e1d None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:23:23.286559 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-7e42b721-153f-4a68-99f4-5f6ce0e22e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:23.345250 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1449/2894] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:23:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:23:23.352656 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731d89c-6593-4b4c-bcac-9b7810b9bda5 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:23.352945 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731d89c-6593-4b4c-bcac-9b7810b9bda5 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:23.353223 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731d89c-6593-4b4c-bcac-9b7810b9bda5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:23.372636 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-6731d89c-6593-4b4c-bcac-9b7810b9bda5 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:23:23.373834 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6731d89c-6593-4b4c-bcac-9b7810b9bda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:23.381807 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1446/2895] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:23:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:23:23.386647 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42b721-153f-4a68-99f4-5f6ce0e22e1d None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:23.396230 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42b721-153f-4a68-99f4-5f6ce0e22e1d None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:23.400848 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63d2e9-fd39-4600-b843-44a1d06bb732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:23.402222 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63d2e9-fd39-4600-b843-44a1d06bb732 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:23.402514 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63d2e9-fd39-4600-b843-44a1d06bb732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:23.402792 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63d2e9-fd39-4600-b843-44a1d06bb732 admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:23.403440 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f63d2e9-fd39-4600-b843-44a1d06bb732 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:23:23.403751 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f63d2e9-fd39-4600-b843-44a1d06bb732 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:23.405389 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f63d2e9-fd39-4600-b843-44a1d06bb732 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:23.416474 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1450/2896] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:23:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:23:23.420619 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731d89c-6593-4b4c-bcac-9b7810b9bda5 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:23.429745 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731d89c-6593-4b4c-bcac-9b7810b9bda5 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:23.435095 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b712f6-eea0-417d-b671-b1f18e769724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:23.435685 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b712f6-eea0-417d-b671-b1f18e769724 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:23.435966 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b712f6-eea0-417d-b671-b1f18e769724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:23.436236 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b712f6-eea0-417d-b671-b1f18e769724 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:23.436914 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b712f6-eea0-417d-b671-b1f18e769724 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:23.438783 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b712f6-eea0-417d-b671-b1f18e769724 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:23.519764 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1447/2897] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:23:23 2022] POST /identity/v3/projects => generated 359 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:23:23.524012 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63d2e9-fd39-4600-b843-44a1d06bb732 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:23.534452 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63d2e9-fd39-4600-b843-44a1d06bb732 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:23.591355 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7614e-ccf2-462b-bde0-f67461230092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:23.591951 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7614e-ccf2-462b-bde0-f67461230092 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:23.592230 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7614e-ccf2-462b-bde0-f67461230092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:23.592602 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7614e-ccf2-462b-bde0-f67461230092 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:23.593235 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7614e-ccf2-462b-bde0-f67461230092 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:23.595164 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7614e-ccf2-462b-bde0-f67461230092 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:23.673125 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1451/2898] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:23:23 2022] POST /identity/v3/users => generated 280 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:23:23.677328 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b712f6-eea0-417d-b671-b1f18e769724 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:23.688289 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b712f6-eea0-417d-b671-b1f18e769724 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:23.692918 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae67a8b3-9b7e-47e4-abc0-c45a56385f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:23.693510 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67a8b3-9b7e-47e4-abc0-c45a56385f1a admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:23.693792 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67a8b3-9b7e-47e4-abc0-c45a56385f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:23.694060 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67a8b3-9b7e-47e4-abc0-c45a56385f1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:23.694688 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae67a8b3-9b7e-47e4-abc0-c45a56385f1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:23.696299 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae67a8b3-9b7e-47e4-abc0-c45a56385f1a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:23.706789 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1448/2899] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:23:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:23:23.711984 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7614e-ccf2-462b-bde0-f67461230092 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:23.722063 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7614e-ccf2-462b-bde0-f67461230092 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:23.726732 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cde095-ee12-43b7-a5d4-bc2a8d3b387b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:23.727321 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cde095-ee12-43b7-a5d4-bc2a8d3b387b admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:23.727606 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cde095-ee12-43b7-a5d4-bc2a8d3b387b admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:23.727880 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cde095-ee12-43b7-a5d4-bc2a8d3b387b admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:23.728559 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cde095-ee12-43b7-a5d4-bc2a8d3b387b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:23.730221 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cde095-ee12-43b7-a5d4-bc2a8d3b387b admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:23.740879 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1452/2900] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:23:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:23:23.745341 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae67a8b3-9b7e-47e4-abc0-c45a56385f1a admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:23.756022 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae67a8b3-9b7e-47e4-abc0-c45a56385f1a admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:23.761794 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972bc78d-d741-479a-b5f2-4011abe816fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:23.762449 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bc78d-d741-479a-b5f2-4011abe816fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:23.762731 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bc78d-d741-479a-b5f2-4011abe816fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:23.763009 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bc78d-d741-479a-b5f2-4011abe816fa admin admin] PATH_INFO: `/v3/projects/b1d66a00dada4b57bb46ca069e8b54ea/users/d1182a03470f44f0b469aef4f21765d2/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:23.774683 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:23:23.777804 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972bc78d-d741-479a-b5f2-4011abe816fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1d66a00dada4b57bb46ca069e8b54ea, user_id=d1182a03470f44f0b469aef4f21765d2, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:23.780560 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972bc78d-d741-479a-b5f2-4011abe816fa admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:23.892746 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1449/2901] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:23:23 2022] PUT /identity/v3/projects/b1d66a00dada4b57bb46ca069e8b54ea/users/d1182a03470f44f0b469aef4f21765d2/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:23:23.903201 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f02c1-90c6-42ff-80c0-8cdf83195ee6 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:23.903486 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f02c1-90c6-42ff-80c0-8cdf83195ee6 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:23.903760 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f02c1-90c6-42ff-80c0-8cdf83195ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:23.917593 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:23:23.933475 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-f72f02c1-90c6-42ff-80c0-8cdf83195ee6 None None] MFA Rules not processed for user `d1182a03470f44f0b469aef4f21765d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:23:23.934605 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-f72f02c1-90c6-42ff-80c0-8cdf83195ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:24.001007 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1453/2902] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:23:23 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 04 01:23:24.037007 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972bc78d-d741-479a-b5f2-4011abe816fa admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:24.038740 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-972bc78d-d741-479a-b5f2-4011abe816fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:24.096409 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972bc78d-d741-479a-b5f2-4011abe816fa admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:24.100165 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd28004-0d41-48c7-bb5d-7cfbd8336d0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:24.100802 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd28004-0d41-48c7-bb5d-7cfbd8336d0a service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:24.101085 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd28004-0d41-48c7-bb5d-7cfbd8336d0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:24.101361 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd28004-0d41-48c7-bb5d-7cfbd8336d0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:24.110269 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd28004-0d41-48c7-bb5d-7cfbd8336d0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:24.111693 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd28004-0d41-48c7-bb5d-7cfbd8336d0a service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:24.121024 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1450/2903] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:23:24 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:23:24.136631 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72f02c1-90c6-42ff-80c0-8cdf83195ee6 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:24.197602 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72f02c1-90c6-42ff-80c0-8cdf83195ee6 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:24.202174 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edacfaa8-6ca8-43c8-bbf5-4c2d7f3387b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:24.202795 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edacfaa8-6ca8-43c8-bbf5-4c2d7f3387b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:24.203071 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edacfaa8-6ca8-43c8-bbf5-4c2d7f3387b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:24.203342 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edacfaa8-6ca8-43c8-bbf5-4c2d7f3387b7 admin admin] PATH_INFO: `/v3/users/d1182a03470f44f0b469aef4f21765d2` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:24.204450 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edacfaa8-6ca8-43c8-bbf5-4c2d7f3387b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1182a03470f44f0b469aef4f21765d2)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:24.206379 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edacfaa8-6ca8-43c8-bbf5-4c2d7f3387b7 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:24.407083 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-edacfaa8-6ca8-43c8-bbf5-4c2d7f3387b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1182a03470f44f0b469aef4f21765d2'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:24.407769 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-edacfaa8-6ca8-43c8-bbf5-4c2d7f3387b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1182a03470f44f0b469aef4f21765d2'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:24.473107 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-edacfaa8-6ca8-43c8-bbf5-4c2d7f3387b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1182a03470f44f0b469aef4f21765d2'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:24.473713 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-edacfaa8-6ca8-43c8-bbf5-4c2d7f3387b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1182a03470f44f0b469aef4f21765d2'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:24.486945 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-edacfaa8-6ca8-43c8-bbf5-4c2d7f3387b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1182a03470f44f0b469aef4f21765d2'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:24.499465 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1454/2904] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:23:24 2022] DELETE /identity/v3/users/d1182a03470f44f0b469aef4f21765d2 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:23:24.504134 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d12f14-011a-404d-ba83-228dd0e43c99 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:24.504425 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d12f14-011a-404d-ba83-228dd0e43c99 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:24.505009 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d12f14-011a-404d-ba83-228dd0e43c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:24.523879 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-97d12f14-011a-404d-ba83-228dd0e43c99 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:23:24.525118 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-97d12f14-011a-404d-ba83-228dd0e43c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:24.583446 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1451/2905] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:23:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:23:24.593656 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edacfaa8-6ca8-43c8-bbf5-4c2d7f3387b7 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:24.595060 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-edacfaa8-6ca8-43c8-bbf5-4c2d7f3387b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:24.612959 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:23:24.652594 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edacfaa8-6ca8-43c8-bbf5-4c2d7f3387b7 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:24.656294 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f95b1d8-078d-4332-875b-843520e4b40e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:24.656933 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95b1d8-078d-4332-875b-843520e4b40e service neutron] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:24.657220 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95b1d8-078d-4332-875b-843520e4b40e service neutron] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:24.657495 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95b1d8-078d-4332-875b-843520e4b40e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:24.666779 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f95b1d8-078d-4332-875b-843520e4b40e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:24.668275 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f95b1d8-078d-4332-875b-843520e4b40e service neutron] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:24.677670 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1455/2906] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:23:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:23:25.070973 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d12f14-011a-404d-ba83-228dd0e43c99 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:25.072394 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-97d12f14-011a-404d-ba83-228dd0e43c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:25.088389 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d12f14-011a-404d-ba83-228dd0e43c99 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:25.093051 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d3245-839f-4232-be7c-ff6b8c9d2a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:25.093684 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d3245-839f-4232-be7c-ff6b8c9d2a97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:25.093960 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d3245-839f-4232-be7c-ff6b8c9d2a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:25.094230 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d3245-839f-4232-be7c-ff6b8c9d2a97 admin admin] PATH_INFO: `/v3/projects/b1d66a00dada4b57bb46ca069e8b54ea` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:25.095340 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82d3245-839f-4232-be7c-ff6b8c9d2a97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1d66a00dada4b57bb46ca069e8b54ea)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:25.097345 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82d3245-839f-4232-be7c-ff6b8c9d2a97 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:25.217521 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f82d3245-839f-4232-be7c-ff6b8c9d2a97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1d66a00dada4b57bb46ca069e8b54ea', 'request_id': 'req-f82d3245-839f-4232-be7c-ff6b8c9d2a97'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:25.275354 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f82d3245-839f-4232-be7c-ff6b8c9d2a97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1d66a00dada4b57bb46ca069e8b54ea', 'request_id': 'req-f82d3245-839f-4232-be7c-ff6b8c9d2a97'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:25.282371 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f82d3245-839f-4232-be7c-ff6b8c9d2a97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1d66a00dada4b57bb46ca069e8b54ea', 'request_id': 'req-f82d3245-839f-4232-be7c-ff6b8c9d2a97'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:25.294944 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f82d3245-839f-4232-be7c-ff6b8c9d2a97 admin admin] The token cache is being invalidate because project b1d66a00dada4b57bb46ca069e8b54ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:23:25.295240 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f82d3245-839f-4232-be7c-ff6b8c9d2a97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:25.295877 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f82d3245-839f-4232-be7c-ff6b8c9d2a97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:25.297394 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1452/2907] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:23:25 2022] DELETE /identity/v3/projects/b1d66a00dada4b57bb46ca069e8b54ea => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:23:25.302213 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c8647-9c59-47a8-a8a3-8d8862cab3e7 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:25.302590 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c8647-9c59-47a8-a8a3-8d8862cab3e7 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:25.302869 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c8647-9c59-47a8-a8a3-8d8862cab3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:25.324361 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-e85c8647-9c59-47a8-a8a3-8d8862cab3e7 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:23:25.325546 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e85c8647-9c59-47a8-a8a3-8d8862cab3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:25.384139 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1456/2908] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:23:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:23:25.392185 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e0c7ba-e5ea-4d12-99c7-3fe30ba9c5a3 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:25.392497 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e0c7ba-e5ea-4d12-99c7-3fe30ba9c5a3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:25.392779 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e0c7ba-e5ea-4d12-99c7-3fe30ba9c5a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:25.413311 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-39e0c7ba-e5ea-4d12-99c7-3fe30ba9c5a3 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:23:25.414502 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-39e0c7ba-e5ea-4d12-99c7-3fe30ba9c5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:25.423373 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1453/2909] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:23:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:23:25.427478 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c8647-9c59-47a8-a8a3-8d8862cab3e7 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:25.437749 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c8647-9c59-47a8-a8a3-8d8862cab3e7 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:25.442436 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a701deb-d96d-4ca6-b800-f3c7394a7463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:25.443040 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a701deb-d96d-4ca6-b800-f3c7394a7463 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:25.443323 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a701deb-d96d-4ca6-b800-f3c7394a7463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:25.443596 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a701deb-d96d-4ca6-b800-f3c7394a7463 admin admin] PATH_INFO: `/v3/domains` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:25.444246 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a701deb-d96d-4ca6-b800-f3c7394a7463 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:23:25.444586 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a701deb-d96d-4ca6-b800-f3c7394a7463 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:25.446189 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a701deb-d96d-4ca6-b800-f3c7394a7463 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:25.457723 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1457/2910] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:23:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:23:25.461741 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e0c7ba-e5ea-4d12-99c7-3fe30ba9c5a3 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:25.471236 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e0c7ba-e5ea-4d12-99c7-3fe30ba9c5a3 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:25.475911 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d591e35-d8a1-4ca9-907e-b7cc316fc8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:25.476541 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d591e35-d8a1-4ca9-907e-b7cc316fc8c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:25.476827 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d591e35-d8a1-4ca9-907e-b7cc316fc8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:25.477104 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d591e35-d8a1-4ca9-907e-b7cc316fc8c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:25.477743 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d591e35-d8a1-4ca9-907e-b7cc316fc8c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:25.480245 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d591e35-d8a1-4ca9-907e-b7cc316fc8c3 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:25.489588 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:23:25.585362 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1454/2911] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:23:25 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:23:25.589731 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a701deb-d96d-4ca6-b800-f3c7394a7463 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:25.599514 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a701deb-d96d-4ca6-b800-f3c7394a7463 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:25.656281 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7841b30c-c020-4a10-b387-53237c8382da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:25.656914 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7841b30c-c020-4a10-b387-53237c8382da admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:25.657195 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7841b30c-c020-4a10-b387-53237c8382da admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:25.657467 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7841b30c-c020-4a10-b387-53237c8382da admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:25.658093 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7841b30c-c020-4a10-b387-53237c8382da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:25.660009 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7841b30c-c020-4a10-b387-53237c8382da admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:25.733221 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1458/2912] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:23:25 2022] POST /identity/v3/users => generated 287 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:23:25.737187 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d591e35-d8a1-4ca9-907e-b7cc316fc8c3 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:25.747765 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d591e35-d8a1-4ca9-907e-b7cc316fc8c3 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:25.752639 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e85d4c-3a96-4500-bad7-ca5c4c634157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:25.753246 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e85d4c-3a96-4500-bad7-ca5c4c634157 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:25.754727 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e85d4c-3a96-4500-bad7-ca5c4c634157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:25.754727 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e85d4c-3a96-4500-bad7-ca5c4c634157 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:25.754727 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e85d4c-3a96-4500-bad7-ca5c4c634157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:25.756305 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e85d4c-3a96-4500-bad7-ca5c4c634157 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:25.766443 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1455/2913] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:23:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:23:25.770451 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7841b30c-c020-4a10-b387-53237c8382da admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:25.781319 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7841b30c-c020-4a10-b387-53237c8382da admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:25.785952 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c576302-505c-4649-b766-24e5e2f7dd75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:25.786525 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c576302-505c-4649-b766-24e5e2f7dd75 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:25.786808 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c576302-505c-4649-b766-24e5e2f7dd75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:25.787082 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c576302-505c-4649-b766-24e5e2f7dd75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:25.787718 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c576302-505c-4649-b766-24e5e2f7dd75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:25.789365 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c576302-505c-4649-b766-24e5e2f7dd75 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:25.800418 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1459/2914] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:23:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 01:23:25.804657 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e85d4c-3a96-4500-bad7-ca5c4c634157 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:25.814869 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e85d4c-3a96-4500-bad7-ca5c4c634157 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:25.819504 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f8ba3-eb6e-4735-b173-bfe696937bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:25.820145 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f8ba3-eb6e-4735-b173-bfe696937bd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:25.820428 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f8ba3-eb6e-4735-b173-bfe696937bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:25.820744 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f8ba3-eb6e-4735-b173-bfe696937bd0 admin admin] PATH_INFO: `/v3/projects/45c594acecfc46db88f67f3fd2b1a30a/users/44075a011a344c69941bad95ee07d2ac/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:25.835217 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f8ba3-eb6e-4735-b173-bfe696937bd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45c594acecfc46db88f67f3fd2b1a30a, user_id=44075a011a344c69941bad95ee07d2ac, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:25.838039 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f8ba3-eb6e-4735-b173-bfe696937bd0 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:25.919560 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1456/2915] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:23:25 2022] PUT /identity/v3/projects/45c594acecfc46db88f67f3fd2b1a30a/users/44075a011a344c69941bad95ee07d2ac/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:23:25.929756 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c19c15-3437-4619-afc7-ee04518ba700 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:25.930048 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c19c15-3437-4619-afc7-ee04518ba700 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:25.930329 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c19c15-3437-4619-afc7-ee04518ba700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:25.957860 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-c4c19c15-3437-4619-afc7-ee04518ba700 None None] MFA Rules not processed for user `44075a011a344c69941bad95ee07d2ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:23:25.959074 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-c4c19c15-3437-4619-afc7-ee04518ba700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:25.971129 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:23:26.023553 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1460/2916] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:23:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:23:26.028637 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f8ba3-eb6e-4735-b173-bfe696937bd0 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:26.091340 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f8ba3-eb6e-4735-b173-bfe696937bd0 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:26.095904 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0aff22-2cbe-4cfc-ab05-e47e93980fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:26.096532 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0aff22-2cbe-4cfc-ab05-e47e93980fcd admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:26.096819 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0aff22-2cbe-4cfc-ab05-e47e93980fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:26.097096 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0aff22-2cbe-4cfc-ab05-e47e93980fcd admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:26.097763 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0aff22-2cbe-4cfc-ab05-e47e93980fcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:26.099654 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0aff22-2cbe-4cfc-ab05-e47e93980fcd admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:26.184843 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1457/2917] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:23:26 2022] POST /identity/v3/projects => generated 375 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:23:26.189005 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c19c15-3437-4619-afc7-ee04518ba700 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:26.199980 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c19c15-3437-4619-afc7-ee04518ba700 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:26.255152 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f8ea5-92b9-4edd-a855-3736b9484218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:26.255748 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f8ea5-92b9-4edd-a855-3736b9484218 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:26.256026 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f8ea5-92b9-4edd-a855-3736b9484218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:26.256296 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f8ea5-92b9-4edd-a855-3736b9484218 admin admin] PATH_INFO: `/v3/users` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:26.256953 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f8ea5-92b9-4edd-a855-3736b9484218 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:26.258854 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f8ea5-92b9-4edd-a855-3736b9484218 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:26.350017 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1461/2918] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:23:26 2022] POST /identity/v3/users => generated 294 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:23:26.354680 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0aff22-2cbe-4cfc-ab05-e47e93980fcd admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:26.364966 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0aff22-2cbe-4cfc-ab05-e47e93980fcd admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:26.369523 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1541d7-5045-49d5-93f7-14e57bcf5d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:26.370698 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1541d7-5045-49d5-93f7-14e57bcf5d62 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:26.370979 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1541d7-5045-49d5-93f7-14e57bcf5d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:26.371326 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1541d7-5045-49d5-93f7-14e57bcf5d62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:26.371960 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1541d7-5045-49d5-93f7-14e57bcf5d62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:26.373571 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1541d7-5045-49d5-93f7-14e57bcf5d62 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:26.384793 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1458/2919] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:23:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:23:26.396373 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f8ea5-92b9-4edd-a855-3736b9484218 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:26.406978 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f8ea5-92b9-4edd-a855-3736b9484218 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:26.411571 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a30c13-0ae6-4fa0-b8d5-2bc1e1ed5057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:26.412214 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a30c13-0ae6-4fa0-b8d5-2bc1e1ed5057 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:26.412527 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a30c13-0ae6-4fa0-b8d5-2bc1e1ed5057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:26.412803 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a30c13-0ae6-4fa0-b8d5-2bc1e1ed5057 admin admin] PATH_INFO: `/v3/projects/5129e7d52fd146dc9ddc4f1cac63de7e/users/b86b0016381247aca39f05bfc17d197c/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:26.427437 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a30c13-0ae6-4fa0-b8d5-2bc1e1ed5057 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5129e7d52fd146dc9ddc4f1cac63de7e, user_id=b86b0016381247aca39f05bfc17d197c, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:26.430309 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a30c13-0ae6-4fa0-b8d5-2bc1e1ed5057 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:26.492006 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1462/2920] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:23:26 2022] PUT /identity/v3/projects/5129e7d52fd146dc9ddc4f1cac63de7e/users/b86b0016381247aca39f05bfc17d197c/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:23:26.496432 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1541d7-5045-49d5-93f7-14e57bcf5d62 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:26.506691 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1541d7-5045-49d5-93f7-14e57bcf5d62 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:26.563969 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ffb59-a6e3-4616-ad1d-a6d19302473e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:26.564594 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ffb59-a6e3-4616-ad1d-a6d19302473e admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:26.564872 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ffb59-a6e3-4616-ad1d-a6d19302473e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:26.565139 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ffb59-a6e3-4616-ad1d-a6d19302473e admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:26.565760 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7ffb59-a6e3-4616-ad1d-a6d19302473e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:26.567409 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7ffb59-a6e3-4616-ad1d-a6d19302473e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:26.577549 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1459/2921] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:23:26 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 04 01:23:26.581682 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a30c13-0ae6-4fa0-b8d5-2bc1e1ed5057 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:26.591767 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a30c13-0ae6-4fa0-b8d5-2bc1e1ed5057 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:26.596278 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f6578-2478-4760-9f58-24b8c87d6b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:26.596946 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f6578-2478-4760-9f58-24b8c87d6b04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:26.597225 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f6578-2478-4760-9f58-24b8c87d6b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:26.597494 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f6578-2478-4760-9f58-24b8c87d6b04 admin admin] PATH_INFO: `/v3/projects/5129e7d52fd146dc9ddc4f1cac63de7e/users/b86b0016381247aca39f05bfc17d197c/roles/c8a59d34ee2f4cefa6940cac778260c7` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:26.599367 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f6578-2478-4760-9f58-24b8c87d6b04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5129e7d52fd146dc9ddc4f1cac63de7e, user_id=b86b0016381247aca39f05bfc17d197c, role_id=c8a59d34ee2f4cefa6940cac778260c7)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:26.602063 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f6578-2478-4760-9f58-24b8c87d6b04 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:26.611923 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1463/2922] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:23:26 2022] PUT /identity/v3/projects/5129e7d52fd146dc9ddc4f1cac63de7e/users/b86b0016381247aca39f05bfc17d197c/roles/c8a59d34ee2f4cefa6940cac778260c7 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:23:26.623442 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b4c13-e906-4321-94e3-81b936fa815b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:26.623731 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b4c13-e906-4321-94e3-81b936fa815b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:26.624010 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b4c13-e906-4321-94e3-81b936fa815b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:26.652000 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-088b4c13-e906-4321-94e3-81b936fa815b None None] MFA Rules not processed for user `b86b0016381247aca39f05bfc17d197c`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:23:26.653237 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-088b4c13-e906-4321-94e3-81b936fa815b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:26.665059 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:23:26.721156 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1460/2923] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:23:26 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:23:26.732381 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f6578-2478-4760-9f58-24b8c87d6b04 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:26.734123 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b27f6578-2478-4760-9f58-24b8c87d6b04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:26.790017 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f6578-2478-4760-9f58-24b8c87d6b04 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:26.794514 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa3637c-1b0b-458b-a361-7b97e3a33325 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:26.794948 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3637c-1b0b-458b-a361-7b97e3a33325 service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:26.795225 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3637c-1b0b-458b-a361-7b97e3a33325 service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:26.795495 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3637c-1b0b-458b-a361-7b97e3a33325 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:26.814056 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:23:26.850591 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa3637c-1b0b-458b-a361-7b97e3a33325 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:26.852022 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa3637c-1b0b-458b-a361-7b97e3a33325 service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:26.861492 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1464/2924] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:23:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:23:30.148993 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b4c13-e906-4321-94e3-81b936fa815b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:30.210464 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b4c13-e906-4321-94e3-81b936fa815b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:30.214845 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690a2e2-81f5-46d8-bd5e-e977c760ee5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:30.215457 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690a2e2-81f5-46d8-bd5e-e977c760ee5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:30.215734 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690a2e2-81f5-46d8-bd5e-e977c760ee5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:30.216005 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690a2e2-81f5-46d8-bd5e-e977c760ee5a admin admin] PATH_INFO: `/v3/users/44075a011a344c69941bad95ee07d2ac` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:30.217106 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7690a2e2-81f5-46d8-bd5e-e977c760ee5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=44075a011a344c69941bad95ee07d2ac)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:30.219003 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7690a2e2-81f5-46d8-bd5e-e977c760ee5a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:30.380946 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7690a2e2-81f5-46d8-bd5e-e977c760ee5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44075a011a344c69941bad95ee07d2ac'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:30.381610 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7690a2e2-81f5-46d8-bd5e-e977c760ee5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44075a011a344c69941bad95ee07d2ac'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:30.389132 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7690a2e2-81f5-46d8-bd5e-e977c760ee5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44075a011a344c69941bad95ee07d2ac'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:30.446764 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7690a2e2-81f5-46d8-bd5e-e977c760ee5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44075a011a344c69941bad95ee07d2ac'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:30.447391 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7690a2e2-81f5-46d8-bd5e-e977c760ee5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44075a011a344c69941bad95ee07d2ac'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:30.466214 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1461/2925] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:23:30 2022] DELETE /identity/v3/users/44075a011a344c69941bad95ee07d2ac => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:23:30.470697 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9f5ca-76bd-44bf-a3cd-173f2764eca4 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:30.471246 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9f5ca-76bd-44bf-a3cd-173f2764eca4 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:30.471246 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9f5ca-76bd-44bf-a3cd-173f2764eca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:30.490157 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-5ef9f5ca-76bd-44bf-a3cd-173f2764eca4 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:23:30.491368 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5ef9f5ca-76bd-44bf-a3cd-173f2764eca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:30.550422 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1465/2926] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:23:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:23:30.560649 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690a2e2-81f5-46d8-bd5e-e977c760ee5a admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:30.562047 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7690a2e2-81f5-46d8-bd5e-e977c760ee5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:30.593102 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:23:30.618677 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690a2e2-81f5-46d8-bd5e-e977c760ee5a admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:30.623803 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff048945-6dbc-44b6-9938-ac7a67053539 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:30.624397 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff048945-6dbc-44b6-9938-ac7a67053539 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:30.624979 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff048945-6dbc-44b6-9938-ac7a67053539 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:30.625975 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff048945-6dbc-44b6-9938-ac7a67053539 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:30.634565 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff048945-6dbc-44b6-9938-ac7a67053539 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:30.636256 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff048945-6dbc-44b6-9938-ac7a67053539 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:30.645621 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1462/2927] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:23:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:23:31.031753 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9f5ca-76bd-44bf-a3cd-173f2764eca4 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:31.033258 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-5ef9f5ca-76bd-44bf-a3cd-173f2764eca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:31.046488 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9f5ca-76bd-44bf-a3cd-173f2764eca4 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:31.051625 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a55689a-2d65-4d39-a71f-7b1d69c82bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:31.052269 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55689a-2d65-4d39-a71f-7b1d69c82bd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:31.052589 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55689a-2d65-4d39-a71f-7b1d69c82bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:31.052870 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55689a-2d65-4d39-a71f-7b1d69c82bd8 admin admin] PATH_INFO: `/v3/projects/45c594acecfc46db88f67f3fd2b1a30a` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:31.054021 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a55689a-2d65-4d39-a71f-7b1d69c82bd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45c594acecfc46db88f67f3fd2b1a30a)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:31.056076 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a55689a-2d65-4d39-a71f-7b1d69c82bd8 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:31.192353 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-7a55689a-2d65-4d39-a71f-7b1d69c82bd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45c594acecfc46db88f67f3fd2b1a30a', 'request_id': 'req-7a55689a-2d65-4d39-a71f-7b1d69c82bd8'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:31.269741 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-7a55689a-2d65-4d39-a71f-7b1d69c82bd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45c594acecfc46db88f67f3fd2b1a30a', 'request_id': 'req-7a55689a-2d65-4d39-a71f-7b1d69c82bd8'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:31.276643 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-7a55689a-2d65-4d39-a71f-7b1d69c82bd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45c594acecfc46db88f67f3fd2b1a30a', 'request_id': 'req-7a55689a-2d65-4d39-a71f-7b1d69c82bd8'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:31.288115 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-7a55689a-2d65-4d39-a71f-7b1d69c82bd8 admin admin] The token cache is being invalidate because project 45c594acecfc46db88f67f3fd2b1a30a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:23:31.288405 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-7a55689a-2d65-4d39-a71f-7b1d69c82bd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:31.289163 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-7a55689a-2d65-4d39-a71f-7b1d69c82bd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:31.292556 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1466/2928] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:23:31 2022] DELETE /identity/v3/projects/45c594acecfc46db88f67f3fd2b1a30a => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:23:31.295504 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff048945-6dbc-44b6-9938-ac7a67053539 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:31.305311 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff048945-6dbc-44b6-9938-ac7a67053539 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:31.365291 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f9c9e5-db68-43cc-8f99-cc321c30355c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:31.365919 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9c9e5-db68-43cc-8f99-cc321c30355c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:31.366195 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9c9e5-db68-43cc-8f99-cc321c30355c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:31.366466 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9c9e5-db68-43cc-8f99-cc321c30355c admin admin] PATH_INFO: `/v3/users/b86b0016381247aca39f05bfc17d197c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:31.367565 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f9c9e5-db68-43cc-8f99-cc321c30355c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b86b0016381247aca39f05bfc17d197c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:31.369568 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f9c9e5-db68-43cc-8f99-cc321c30355c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:31.546655 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-39f9c9e5-db68-43cc-8f99-cc321c30355c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b86b0016381247aca39f05bfc17d197c'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:31.547269 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-39f9c9e5-db68-43cc-8f99-cc321c30355c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b86b0016381247aca39f05bfc17d197c'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:31.554172 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-39f9c9e5-db68-43cc-8f99-cc321c30355c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b86b0016381247aca39f05bfc17d197c'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:31.627499 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-39f9c9e5-db68-43cc-8f99-cc321c30355c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b86b0016381247aca39f05bfc17d197c'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:31.628157 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-39f9c9e5-db68-43cc-8f99-cc321c30355c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b86b0016381247aca39f05bfc17d197c'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:31.646265 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1463/2929] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:23:31 2022] DELETE /identity/v3/users/b86b0016381247aca39f05bfc17d197c => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:23:32.031849 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a55689a-2d65-4d39-a71f-7b1d69c82bd8 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:32.033349 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-7a55689a-2d65-4d39-a71f-7b1d69c82bd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:32.046522 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a55689a-2d65-4d39-a71f-7b1d69c82bd8 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:32.091686 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:23:32.106394 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390bd607-d385-4edc-a2c8-286faa26d519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:32.107043 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bd607-d385-4edc-a2c8-286faa26d519 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:32.107310 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bd607-d385-4edc-a2c8-286faa26d519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:32.107587 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bd607-d385-4edc-a2c8-286faa26d519 admin admin] PATH_INFO: `/v3/projects/5129e7d52fd146dc9ddc4f1cac63de7e` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:32.108743 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390bd607-d385-4edc-a2c8-286faa26d519 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5129e7d52fd146dc9ddc4f1cac63de7e)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:32.114451 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390bd607-d385-4edc-a2c8-286faa26d519 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:32.253238 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-390bd607-d385-4edc-a2c8-286faa26d519 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5129e7d52fd146dc9ddc4f1cac63de7e', 'request_id': 'req-390bd607-d385-4edc-a2c8-286faa26d519'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:32.314886 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-390bd607-d385-4edc-a2c8-286faa26d519 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5129e7d52fd146dc9ddc4f1cac63de7e', 'request_id': 'req-390bd607-d385-4edc-a2c8-286faa26d519'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:32.320698 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-390bd607-d385-4edc-a2c8-286faa26d519 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5129e7d52fd146dc9ddc4f1cac63de7e', 'request_id': 'req-390bd607-d385-4edc-a2c8-286faa26d519'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:32.332630 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-390bd607-d385-4edc-a2c8-286faa26d519 admin admin] The token cache is being invalidate because project 5129e7d52fd146dc9ddc4f1cac63de7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:23:32.332952 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-390bd607-d385-4edc-a2c8-286faa26d519 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:32.333584 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-390bd607-d385-4edc-a2c8-286faa26d519 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:23:32.335059 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1467/2930] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:23:32 2022] DELETE /identity/v3/projects/5129e7d52fd146dc9ddc4f1cac63de7e => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:23:32.339932 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527d779-c716-4006-8026-d5f766f14b1b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:32.341231 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527d779-c716-4006-8026-d5f766f14b1b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:32.341231 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527d779-c716-4006-8026-d5f766f14b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:32.360867 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-6527d779-c716-4006-8026-d5f766f14b1b None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:23:32.362052 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6527d779-c716-4006-8026-d5f766f14b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:32.406144 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:23:32.419812 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1464/2931] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:23:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:23:32.427573 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f4242-f806-4160-84c6-4510d21b938e None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:32.427858 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f4242-f806-4160-84c6-4510d21b938e None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:32.428132 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f4242-f806-4160-84c6-4510d21b938e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:32.449229 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-474f4242-f806-4160-84c6-4510d21b938e None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:23:32.452505 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-474f4242-f806-4160-84c6-4510d21b938e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:32.459495 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1468/2932] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:23:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:23:32.463766 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527d779-c716-4006-8026-d5f766f14b1b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:32.474507 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527d779-c716-4006-8026-d5f766f14b1b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:32.479022 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736168e-abde-4142-8771-445dcef9047e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:32.479612 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736168e-abde-4142-8771-445dcef9047e admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:32.479887 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736168e-abde-4142-8771-445dcef9047e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:32.480157 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736168e-abde-4142-8771-445dcef9047e admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:32.480847 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736168e-abde-4142-8771-445dcef9047e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:23:32.481150 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736168e-abde-4142-8771-445dcef9047e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:32.482880 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736168e-abde-4142-8771-445dcef9047e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:32.494914 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1465/2933] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:23:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:23:32.499907 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474f4242-f806-4160-84c6-4510d21b938e None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:32.509388 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474f4242-f806-4160-84c6-4510d21b938e None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:32.514705 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0cbe06-26fb-427d-b093-1a966fd8bb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:32.515293 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0cbe06-26fb-427d-b093-1a966fd8bb23 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:32.515568 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0cbe06-26fb-427d-b093-1a966fd8bb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:32.515839 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0cbe06-26fb-427d-b093-1a966fd8bb23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:32.516479 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0cbe06-26fb-427d-b093-1a966fd8bb23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:32.518375 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0cbe06-26fb-427d-b093-1a966fd8bb23 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:32.631611 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1469/2934] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:23:32 2022] POST /identity/v3/projects => generated 369 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:23:32.635780 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736168e-abde-4142-8771-445dcef9047e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:32.646500 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736168e-abde-4142-8771-445dcef9047e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:32.703659 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7b8c4-eecd-43ea-959d-705793a6093f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:32.704253 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7b8c4-eecd-43ea-959d-705793a6093f admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:32.704561 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7b8c4-eecd-43ea-959d-705793a6093f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:32.704834 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7b8c4-eecd-43ea-959d-705793a6093f admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:32.705459 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7b8c4-eecd-43ea-959d-705793a6093f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:32.707367 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7b8c4-eecd-43ea-959d-705793a6093f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:32.833522 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1466/2935] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:23:32 2022] POST /identity/v3/users => generated 285 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 04 01:23:32.834704 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0cbe06-26fb-427d-b093-1a966fd8bb23 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:32.845169 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0cbe06-26fb-427d-b093-1a966fd8bb23 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:32.849723 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baace5d-7242-47bb-b827-3a4c6cb33457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:32.850314 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baace5d-7242-47bb-b827-3a4c6cb33457 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:32.850613 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baace5d-7242-47bb-b827-3a4c6cb33457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:32.850871 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baace5d-7242-47bb-b827-3a4c6cb33457 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:32.851696 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baace5d-7242-47bb-b827-3a4c6cb33457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:32.853350 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baace5d-7242-47bb-b827-3a4c6cb33457 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:32.864398 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1470/2936] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:23:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:23:32.868363 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7b8c4-eecd-43ea-959d-705793a6093f admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:32.879052 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7b8c4-eecd-43ea-959d-705793a6093f admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:32.884070 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed74bf-0a04-4aeb-86b9-4d60926d136c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:32.885261 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed74bf-0a04-4aeb-86b9-4d60926d136c admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:32.885551 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed74bf-0a04-4aeb-86b9-4d60926d136c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:32.885832 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed74bf-0a04-4aeb-86b9-4d60926d136c admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:32.886473 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed74bf-0a04-4aeb-86b9-4d60926d136c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:32.888060 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed74bf-0a04-4aeb-86b9-4d60926d136c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:32.898719 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1467/2937] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:23:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:23:32.902980 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baace5d-7242-47bb-b827-3a4c6cb33457 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:32.912996 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baace5d-7242-47bb-b827-3a4c6cb33457 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:32.917573 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339b61f-7a70-46e8-80ab-58754769702e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:32.924482 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339b61f-7a70-46e8-80ab-58754769702e admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:32.924482 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339b61f-7a70-46e8-80ab-58754769702e admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:32.924482 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339b61f-7a70-46e8-80ab-58754769702e admin admin] PATH_INFO: `/v3/projects/450c038301c24b58b41e09adb13cba09/users/6081214e9ce44092974b812de0d5c120/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:32.934556 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a339b61f-7a70-46e8-80ab-58754769702e admin admin] RBAC: Authorizing `identity:create_grant(project_id=450c038301c24b58b41e09adb13cba09, user_id=6081214e9ce44092974b812de0d5c120, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:32.937460 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a339b61f-7a70-46e8-80ab-58754769702e admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:33.012517 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1471/2938] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:23:32 2022] PUT /identity/v3/projects/450c038301c24b58b41e09adb13cba09/users/6081214e9ce44092974b812de0d5c120/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:23:33.022877 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34f846-e416-4a51-9b71-8004cca0b15d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:33.026004 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34f846-e416-4a51-9b71-8004cca0b15d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:33.026004 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34f846-e416-4a51-9b71-8004cca0b15d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:33.051633 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-3b34f846-e416-4a51-9b71-8004cca0b15d None None] MFA Rules not processed for user `6081214e9ce44092974b812de0d5c120`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:23:33.052885 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3b34f846-e416-4a51-9b71-8004cca0b15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:33.116723 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1468/2939] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:23:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:23:33.129064 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339b61f-7a70-46e8-80ab-58754769702e admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:23:33.131080 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-a339b61f-7a70-46e8-80ab-58754769702e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:23:33.208723 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339b61f-7a70-46e8-80ab-58754769702e admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:23:33.212333 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf054a-2aa2-43aa-b030-0f1b27651fe4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1179503bdab547c99175a27b7ce27b9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:23:33.212964 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf054a-2aa2-43aa-b030-0f1b27651fe4 service glance] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:23:33.213244 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf054a-2aa2-43aa-b030-0f1b27651fe4 service glance] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:23:33.213517 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf054a-2aa2-43aa-b030-0f1b27651fe4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:23:33.219409 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:23:33.223142 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cf054a-2aa2-43aa-b030-0f1b27651fe4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:23:33.224701 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cf054a-2aa2-43aa-b030-0f1b27651fe4 service glance] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:23:33.234588 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1472/2940] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:23:33 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:23:43.113820 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=76,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:23:43.309286 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:24:01.585885 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34f846-e416-4a51-9b71-8004cca0b15d None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:24:01.647984 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34f846-e416-4a51-9b71-8004cca0b15d None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:24:01.652785 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9399075e-e8d4-478a-a796-9044b736d820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:24:01.653399 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9399075e-e8d4-478a-a796-9044b736d820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:24:01.653676 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9399075e-e8d4-478a-a796-9044b736d820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:24:01.653947 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9399075e-e8d4-478a-a796-9044b736d820 admin admin] PATH_INFO: `/v3/users/6081214e9ce44092974b812de0d5c120` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:24:01.655023 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9399075e-e8d4-478a-a796-9044b736d820 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6081214e9ce44092974b812de0d5c120)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:24:01.656951 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9399075e-e8d4-478a-a796-9044b736d820 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:24:01.830898 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9399075e-e8d4-478a-a796-9044b736d820 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6081214e9ce44092974b812de0d5c120'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:24:01.831474 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9399075e-e8d4-478a-a796-9044b736d820 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6081214e9ce44092974b812de0d5c120'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:24:01.839719 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9399075e-e8d4-478a-a796-9044b736d820 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6081214e9ce44092974b812de0d5c120'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:24:01.889698 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9399075e-e8d4-478a-a796-9044b736d820 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6081214e9ce44092974b812de0d5c120'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:24:01.890314 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9399075e-e8d4-478a-a796-9044b736d820 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6081214e9ce44092974b812de0d5c120'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:24:01.908581 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1469/2941] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:24:01 2022] DELETE /identity/v3/users/6081214e9ce44092974b812de0d5c120 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:24:01.913112 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f5136-1995-4ecc-93bd-2e5d803b7a1d None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:24:01.913571 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f5136-1995-4ecc-93bd-2e5d803b7a1d None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:24:01.913693 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f5136-1995-4ecc-93bd-2e5d803b7a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:24:01.933459 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-051f5136-1995-4ecc-93bd-2e5d803b7a1d None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:24:01.934717 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-051f5136-1995-4ecc-93bd-2e5d803b7a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:24:01.993080 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1473/2942] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:24:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:24:02.003031 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9399075e-e8d4-478a-a796-9044b736d820 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:24:02.004746 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9399075e-e8d4-478a-a796-9044b736d820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:24:02.061783 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9399075e-e8d4-478a-a796-9044b736d820 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:24:02.065434 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1b90d-471c-40f7-80d5-a580050a4621 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:24:02.066035 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1b90d-471c-40f7-80d5-a580050a4621 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:24:02.066316 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1b90d-471c-40f7-80d5-a580050a4621 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:24:02.066591 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1b90d-471c-40f7-80d5-a580050a4621 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:24:02.075523 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df1b90d-471c-40f7-80d5-a580050a4621 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:24:02.077003 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df1b90d-471c-40f7-80d5-a580050a4621 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:24:02.086225 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1470/2943] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:24:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 04 01:24:02.494271 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f5136-1995-4ecc-93bd-2e5d803b7a1d None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:24:02.495875 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-051f5136-1995-4ecc-93bd-2e5d803b7a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:24:02.507847 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f5136-1995-4ecc-93bd-2e5d803b7a1d None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:24:02.513775 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd5e2e-84e0-4a0e-9063-59f2f1360a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:24:02.514435 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd5e2e-84e0-4a0e-9063-59f2f1360a4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:24:02.514715 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd5e2e-84e0-4a0e-9063-59f2f1360a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:24:02.514990 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd5e2e-84e0-4a0e-9063-59f2f1360a4b admin admin] PATH_INFO: `/v3/projects/450c038301c24b58b41e09adb13cba09` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:24:02.516122 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dd5e2e-84e0-4a0e-9063-59f2f1360a4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=450c038301c24b58b41e09adb13cba09)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:24:02.518076 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dd5e2e-84e0-4a0e-9063-59f2f1360a4b admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:24:02.656032 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-45dd5e2e-84e0-4a0e-9063-59f2f1360a4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '450c038301c24b58b41e09adb13cba09', 'request_id': 'req-45dd5e2e-84e0-4a0e-9063-59f2f1360a4b'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:24:02.730092 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-45dd5e2e-84e0-4a0e-9063-59f2f1360a4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '450c038301c24b58b41e09adb13cba09', 'request_id': 'req-45dd5e2e-84e0-4a0e-9063-59f2f1360a4b'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:24:02.735846 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-45dd5e2e-84e0-4a0e-9063-59f2f1360a4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '450c038301c24b58b41e09adb13cba09', 'request_id': 'req-45dd5e2e-84e0-4a0e-9063-59f2f1360a4b'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:24:02.747781 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-45dd5e2e-84e0-4a0e-9063-59f2f1360a4b admin admin] The token cache is being invalidate because project 450c038301c24b58b41e09adb13cba09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:24:02.751819 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-45dd5e2e-84e0-4a0e-9063-59f2f1360a4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:24:02.751819 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-45dd5e2e-84e0-4a0e-9063-59f2f1360a4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:24:02.751819 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1474/2944] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:24:02 2022] DELETE /identity/v3/projects/450c038301c24b58b41e09adb13cba09 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:24:02.755418 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f333285-68ff-4e83-8ea1-c8d927f7c65c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:24:02.755888 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f333285-68ff-4e83-8ea1-c8d927f7c65c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:24:02.756004 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f333285-68ff-4e83-8ea1-c8d927f7c65c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:24:02.776361 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-4f333285-68ff-4e83-8ea1-c8d927f7c65c None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:24:02.777589 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4f333285-68ff-4e83-8ea1-c8d927f7c65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:24:02.794813 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:24:02.838379 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1471/2945] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:24:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:24:02.846683 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aad69f-eb0f-4f2a-bd08-153b67be4b07 None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:24:02.849483 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aad69f-eb0f-4f2a-bd08-153b67be4b07 None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:24:02.849483 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aad69f-eb0f-4f2a-bd08-153b67be4b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:24:02.867488 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-e0aad69f-eb0f-4f2a-bd08-153b67be4b07 None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:24:02.869553 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-e0aad69f-eb0f-4f2a-bd08-153b67be4b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:24:02.877329 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1475/2946] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:24:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:24:02.881860 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f333285-68ff-4e83-8ea1-c8d927f7c65c None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:24:02.891677 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f333285-68ff-4e83-8ea1-c8d927f7c65c None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:24:02.896424 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8641df5e-aa74-41fb-a665-42a8a77c4097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:24:02.897057 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641df5e-aa74-41fb-a665-42a8a77c4097 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:24:02.897338 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641df5e-aa74-41fb-a665-42a8a77c4097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:24:02.897609 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641df5e-aa74-41fb-a665-42a8a77c4097 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:24:02.898247 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8641df5e-aa74-41fb-a665-42a8a77c4097 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 04 01:24:02.898547 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8641df5e-aa74-41fb-a665-42a8a77c4097 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:24:02.900112 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8641df5e-aa74-41fb-a665-42a8a77c4097 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:24:02.911809 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1472/2947] 10.222.0.27 () {58 vars in 1153 bytes} [Sun Sep 4 01:24:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 04 01:24:02.916653 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aad69f-eb0f-4f2a-bd08-153b67be4b07 None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:24:02.926360 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aad69f-eb0f-4f2a-bd08-153b67be4b07 None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:24:02.931788 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2606b-705a-4421-874a-3d2e998be1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:24:02.932383 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2606b-705a-4421-874a-3d2e998be1d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:24:02.932699 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2606b-705a-4421-874a-3d2e998be1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:24:02.932973 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2606b-705a-4421-874a-3d2e998be1d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:24:02.933599 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e2606b-705a-4421-874a-3d2e998be1d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:24:02.935494 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e2606b-705a-4421-874a-3d2e998be1d6 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:24:03.043090 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1476/2948] 10.222.0.27 () {60 vars in 1153 bytes} [Sun Sep 4 01:24:02 2022] POST /identity/v3/projects => generated 365 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:24:03.046939 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8641df5e-aa74-41fb-a665-42a8a77c4097 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:24:03.059134 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8641df5e-aa74-41fb-a665-42a8a77c4097 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:24:03.116970 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1b08de-03e2-4945-b337-c10e697ccb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:24:03.117569 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b08de-03e2-4945-b337-c10e697ccb97 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:24:03.117847 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b08de-03e2-4945-b337-c10e697ccb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:24:03.118120 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b08de-03e2-4945-b337-c10e697ccb97 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:24:03.119396 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1b08de-03e2-4945-b337-c10e697ccb97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:24:03.121418 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1b08de-03e2-4945-b337-c10e697ccb97 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:24:03.224135 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1473/2949] 10.222.0.27 () {60 vars in 1144 bytes} [Sun Sep 4 01:24:03 2022] POST /identity/v3/users => generated 283 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 04 01:24:03.229177 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2606b-705a-4421-874a-3d2e998be1d6 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:24:03.241561 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2606b-705a-4421-874a-3d2e998be1d6 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:24:03.247462 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c1ebe-a86c-4fcd-8241-0dfdb3e9d356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:24:03.248061 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c1ebe-a86c-4fcd-8241-0dfdb3e9d356 admin admin] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:24:03.248498 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c1ebe-a86c-4fcd-8241-0dfdb3e9d356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:24:03.250716 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c1ebe-a86c-4fcd-8241-0dfdb3e9d356 admin admin] PATH_INFO: `/v3/roles` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:24:03.251370 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4c1ebe-a86c-4fcd-8241-0dfdb3e9d356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:24:03.255269 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4c1ebe-a86c-4fcd-8241-0dfdb3e9d356 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:24:03.267990 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1477/2950] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:24:03.273983 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1b08de-03e2-4945-b337-c10e697ccb97 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:24:03.287087 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1b08de-03e2-4945-b337-c10e697ccb97 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:24:03.293053 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f82e9f-88fd-4e66-9862-44074fd3cf5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:24:03.293744 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f82e9f-88fd-4e66-9862-44074fd3cf5b admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:24:03.294114 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f82e9f-88fd-4e66-9862-44074fd3cf5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:24:03.294546 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f82e9f-88fd-4e66-9862-44074fd3cf5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:24:03.295486 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f82e9f-88fd-4e66-9862-44074fd3cf5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:24:03.297400 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f82e9f-88fd-4e66-9862-44074fd3cf5b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:24:03.308525 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1474/2951] 10.222.0.27 () {58 vars in 1122 bytes} [Sun Sep 4 01:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 04 01:24:03.313017 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c1ebe-a86c-4fcd-8241-0dfdb3e9d356 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:24:03.326187 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c1ebe-a86c-4fcd-8241-0dfdb3e9d356 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:24:03.332252 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c6460-9747-44aa-bba2-2dc5998254d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:24:03.332941 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c6460-9747-44aa-bba2-2dc5998254d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:24:03.333224 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c6460-9747-44aa-bba2-2dc5998254d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:24:03.333499 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c6460-9747-44aa-bba2-2dc5998254d3 admin admin] PATH_INFO: `/v3/projects/9ac8e18a7c084997b9f60f41e1cbb93e/users/f49829dc0635458eb688f6ed4e6bb3b8/roles/d1261c99e1d64f9da63adfff739561e8` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:24:03.350064 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c6460-9747-44aa-bba2-2dc5998254d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ac8e18a7c084997b9f60f41e1cbb93e, user_id=f49829dc0635458eb688f6ed4e6bb3b8, role_id=d1261c99e1d64f9da63adfff739561e8)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:24:03.353436 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c6460-9747-44aa-bba2-2dc5998254d3 admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:24:03.424167 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1478/2952] 10.222.0.27 () {60 vars in 1483 bytes} [Sun Sep 4 01:24:03 2022] PUT /identity/v3/projects/9ac8e18a7c084997b9f60f41e1cbb93e/users/f49829dc0635458eb688f6ed4e6bb3b8/roles/d1261c99e1d64f9da63adfff739561e8 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:24:03.438718 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419100bb-52ff-4000-afc5-b5e681d0710d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:24:03.439004 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419100bb-52ff-4000-afc5-b5e681d0710d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:24:03.439258 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419100bb-52ff-4000-afc5-b5e681d0710d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:24:03.474296 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-419100bb-52ff-4000-afc5-b5e681d0710d None None] MFA Rules not processed for user `f49829dc0635458eb688f6ed4e6bb3b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:24:03.475422 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-419100bb-52ff-4000-afc5-b5e681d0710d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:24:03.548877 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1475/2953] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:24:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:24:03.560753 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c6460-9747-44aa-bba2-2dc5998254d3 admin admin] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:24:03.562463 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-019c6460-9747-44aa-bba2-2dc5998254d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:24:03.585365 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:24:03.624373 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c6460-9747-44aa-bba2-2dc5998254d3 admin admin] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:24:03.628093 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea80b152-17a1-4827-b689-18c79abdbf9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6297dbc521e94330ae9294b942144181', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:24:03.629251 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80b152-17a1-4827-b689-18c79abdbf9a service cinder] REQUEST_METHOD: `GET` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:24:03.629603 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80b152-17a1-4827-b689-18c79abdbf9a service cinder] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:24:03.630096 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80b152-17a1-4827-b689-18c79abdbf9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:24:03.641718 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea80b152-17a1-4827-b689-18c79abdbf9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:24:03.646696 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea80b152-17a1-4827-b689-18c79abdbf9a service cinder] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:24:03.653186 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1479/2954] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:24:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:24:13.545925 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:24:13.686054 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=17 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:24:41.599392 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419100bb-52ff-4000-afc5-b5e681d0710d None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:24:41.712019 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419100bb-52ff-4000-afc5-b5e681d0710d None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:24:41.716544 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8c13c-b5d4-46d6-8007-5877083ddc37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:24:41.717161 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8c13c-b5d4-46d6-8007-5877083ddc37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:24:41.717439 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8c13c-b5d4-46d6-8007-5877083ddc37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:24:41.717708 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8c13c-b5d4-46d6-8007-5877083ddc37 admin admin] PATH_INFO: `/v3/users/f49829dc0635458eb688f6ed4e6bb3b8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:24:41.718809 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f8c13c-b5d4-46d6-8007-5877083ddc37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f49829dc0635458eb688f6ed4e6bb3b8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:24:41.720725 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f8c13c-b5d4-46d6-8007-5877083ddc37 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:24:41.898831 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-27f8c13c-b5d4-46d6-8007-5877083ddc37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f49829dc0635458eb688f6ed4e6bb3b8'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:24:41.899414 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-27f8c13c-b5d4-46d6-8007-5877083ddc37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f49829dc0635458eb688f6ed4e6bb3b8'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:24:41.906519 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-27f8c13c-b5d4-46d6-8007-5877083ddc37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f49829dc0635458eb688f6ed4e6bb3b8'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:24:41.965604 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-27f8c13c-b5d4-46d6-8007-5877083ddc37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f49829dc0635458eb688f6ed4e6bb3b8'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:24:41.966206 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-27f8c13c-b5d4-46d6-8007-5877083ddc37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f49829dc0635458eb688f6ed4e6bb3b8'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:24:41.984402 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1476/2955] 10.222.0.27 () {58 vars in 1224 bytes} [Sun Sep 4 01:24:41 2022] DELETE /identity/v3/users/f49829dc0635458eb688f6ed4e6bb3b8 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 04 01:24:41.989809 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5406a-d7f4-4f5c-9e02-082e3f7d1ddc None None] REQUEST_METHOD: `POST` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:24:41.990344 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5406a-d7f4-4f5c-9e02-082e3f7d1ddc None None] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:24:41.990507 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5406a-d7f4-4f5c-9e02-082e3f7d1ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:24:42.040230 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.auth.core [None req-b3d5406a-d7f4-4f5c-9e02-082e3f7d1ddc None None] MFA Rules not processed for user `644f0b1b120742d2aca15aeeaa76abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 04 01:24:42.041468 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b3d5406a-d7f4-4f5c-9e02-082e3f7d1ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:24:42.098828 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1480/2956] 10.222.0.27 () {58 vars in 958 bytes} [Sun Sep 4 01:24:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 04 01:24:42.108411 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8c13c-b5d4-46d6-8007-5877083ddc37 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:24:42.109894 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-27f8c13c-b5d4-46d6-8007-5877083ddc37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:24:42.152845 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:24:42.184084 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8c13c-b5d4-46d6-8007-5877083ddc37 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:24:42.187786 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5db64-c92e-48f2-9e5c-558a6f790763 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '861c278f8bf34b3bb407b2e051119ee9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:24:42.188386 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5db64-c92e-48f2-9e5c-558a6f790763 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:24:42.188702 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5db64-c92e-48f2-9e5c-558a6f790763 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:24:42.188982 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5db64-c92e-48f2-9e5c-558a6f790763 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:24:42.199288 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e5db64-c92e-48f2-9e5c-558a6f790763 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:24:42.200798 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e5db64-c92e-48f2-9e5c-558a6f790763 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:24:42.210192 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1477/2957] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:24:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:24:42.628942 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d5406a-d7f4-4f5c-9e02-082e3f7d1ddc None None] Authenticating user token {{(pid=81630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:24:42.630492 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.fernet_utils [None req-b3d5406a-d7f4-4f5c-9e02-082e3f7d1ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:24:42.643892 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d5406a-d7f4-4f5c-9e02-082e3f7d1ddc None None] Validating token access rules against request {{(pid=81630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:24:42.648973 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24695f-9501-4fd6-a902-17c6f66d6afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '644f0b1b120742d2aca15aeeaa76abcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1d8c40dd6c04e8e8fe6b0c2b34723cd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:24:42.649208 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24695f-9501-4fd6-a902-17c6f66d6afc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:24:42.649492 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24695f-9501-4fd6-a902-17c6f66d6afc admin admin] SCRIPT_NAME: `/identity` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:24:42.649769 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24695f-9501-4fd6-a902-17c6f66d6afc admin admin] PATH_INFO: `/v3/projects/9ac8e18a7c084997b9f60f41e1cbb93e` {{(pid=81630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:24:42.650887 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd24695f-9501-4fd6-a902-17c6f66d6afc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ac8e18a7c084997b9f60f41e1cbb93e)` {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:24:42.652846 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd24695f-9501-4fd6-a902-17c6f66d6afc admin admin] RBAC: Authorization granted {{(pid=81630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:24:42.778189 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-cd24695f-9501-4fd6-a902-17c6f66d6afc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ac8e18a7c084997b9f60f41e1cbb93e', 'request_id': 'req-cd24695f-9501-4fd6-a902-17c6f66d6afc'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:24:42.840153 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-cd24695f-9501-4fd6-a902-17c6f66d6afc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ac8e18a7c084997b9f60f41e1cbb93e', 'request_id': 'req-cd24695f-9501-4fd6-a902-17c6f66d6afc'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:24:42.846441 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-cd24695f-9501-4fd6-a902-17c6f66d6afc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ac8e18a7c084997b9f60f41e1cbb93e', 'request_id': 'req-cd24695f-9501-4fd6-a902-17c6f66d6afc'} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:24:42.858499 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-cd24695f-9501-4fd6-a902-17c6f66d6afc admin admin] The token cache is being invalidate because project 9ac8e18a7c084997b9f60f41e1cbb93e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 04 01:24:42.858788 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-cd24695f-9501-4fd6-a902-17c6f66d6afc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:24:42.859418 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG keystone.notifications [None req-cd24695f-9501-4fd6-a902-17c6f66d6afc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 04 01:24:42.860965 dv-cinder-master-3-smb devstack@keystone.service[81630]: [pid: 81630|app: 0|req: 1481/2958] 10.222.0.27 () {58 vars in 1233 bytes} [Sun Sep 4 01:24:42 2022] DELETE /identity/v3/projects/9ac8e18a7c084997b9f60f41e1cbb93e => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 04 01:24:52.249614 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=16 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:24:52.855548 dv-cinder-master-3-smb devstack@keystone.service[81630]: DEBUG dbcounter [-] [81630] Writing DB stats keystone:SELECT=66,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81630) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:25:45.870890 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5db64-c92e-48f2-9e5c-558a6f790763 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 04 01:25:45.872322 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-59e5db64-c92e-48f2-9e5c-558a6f790763 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:25:45.962856 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5db64-c92e-48f2-9e5c-558a6f790763 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 04 01:25:46.021389 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47842cc9-3b3b-4fd4-b835-ab9432c0101d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '018802a8d9c041119ad708430f885a89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5100a8d316d243fb8f82e9044e1b3b06', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 04 01:25:46.021990 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47842cc9-3b3b-4fd4-b835-ab9432c0101d service placement] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 04 01:25:46.022268 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47842cc9-3b3b-4fd4-b835-ab9432c0101d service placement] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 04 01:25:46.022538 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47842cc9-3b3b-4fd4-b835-ab9432c0101d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 04 01:25:46.024251 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-47842cc9-3b3b-4fd4-b835-ab9432c0101d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 04 01:25:46.035099 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47842cc9-3b3b-4fd4-b835-ab9432c0101d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 04 01:25:46.036678 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47842cc9-3b3b-4fd4-b835-ab9432c0101d service placement] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 04 01:25:46.047084 dv-cinder-master-3-smb devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1478/2959] 10.222.0.27 () {60 vars in 1365 bytes} [Sun Sep 4 01:25:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 04 01:25:56.042878 dv-cinder-master-3-smb devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=67 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}